-- Logs begin at Thu 2021-05-20 03:41:47 UTC, end at Thu 2021-05-20 04:43:00 UTC. -- May 20 04:13:56.876895 dv-cinder-782405-9-iscsi systemd[1]: Started Devstack devstack@q-svc.service. May 20 04:13:59.256689 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.common.config [-] Logging enabled! May 20 04:13:59.257299 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.common.config [-] /usr/local/bin/neutron-server version 18.1.0.dev214 May 20 04:13:59.257427 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=91536) setup_logging /opt/stack/neutron/neutron/common/config.py:112}} May 20 04:13:59.260599 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.common.config [-] Logging enabled! May 20 04:13:59.260854 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.common.config [-] /usr/local/bin/neutron-server version 18.1.0.dev214 May 20 04:13:59.261167 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=91536) setup_logging /opt/stack/neutron/neutron/common/config.py:112}} May 20 04:13:59.262096 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ******************************************************************************** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 20 04:13:59.262325 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] Configuration options gathered from: {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 20 04:13:59.262690 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 20 04:13:59.262934 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 20 04:13:59.263161 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ================================================================================ {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 20 04:13:59.263509 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] allow_bulk = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.263810 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] allow_overlapping_ips = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.264110 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] api_extensions_path = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.264323 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] api_paste_config = api-paste.ini {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.264617 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] api_workers = 2 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.264901 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] auth_strategy = keystone {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.265182 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] backlog = 4096 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.265469 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] base_mac = fa:16:3e:00:00:00 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.265769 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] bind_host = 0.0.0.0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.266050 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] bind_port = 9696 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.266333 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] client_socket_timeout = 900 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.266598 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] config_dir = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.266827 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] config_file = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.267108 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] config_source = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.267524 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] control_exchange = neutron {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.267809 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] core_plugin = ml2 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.268035 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] debug = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.268313 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] default_availability_zones = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.268736 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.269154 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] dhcp_agent_notification = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.269433 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] dhcp_lease_duration = 86400 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.269724 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] dns_domain = openstacklocal {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.270002 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] enable_traditional_dhcp = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.270272 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] external_dns_driver = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.270548 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] filter_validation = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.270830 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] global_physnet_mtu = 1500 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.271156 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] host = dv-cinder-782405-9-iscsi {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.271459 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] http_retries = 3 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.271751 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] instance_format = [instance: %(uuid)s] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.272040 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.272321 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ipam_driver = internal {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.272597 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ipv6_pd_enabled = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.272870 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] log_config_append = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.273155 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.273427 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] log_dir = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.273699 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] log_file = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.273971 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] log_rotate_interval = 1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.274262 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] log_rotate_interval_type = days {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.274548 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] log_rotation_type = none {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.274840 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.275064 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.275315 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.275610 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.275897 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.276171 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] max_dns_nameservers = 5 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.276444 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] max_header_line = 16384 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.276718 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] max_logfile_count = 30 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.276992 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] max_logfile_size_mb = 200 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.277265 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] max_subnet_host_routes = 20 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.277544 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] network_link_prefix = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.277818 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] notify_nova_on_port_data_changes = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.278102 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] notify_nova_on_port_status_changes = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.278385 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] pagination_max_limit = -1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.278657 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] periodic_fuzzy_delay = 5 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.278928 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] periodic_interval = 40 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.279201 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] publish_errors = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.279493 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] rate_limit_burst = 0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.279778 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] rate_limit_except_level = CRITICAL {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.280054 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] rate_limit_interval = 0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.280327 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] retry_until_window = 30 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.280600 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] rpc_response_max_timeout = 600 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.280885 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] rpc_state_report_workers = 0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.281153 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] rpc_workers = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.281427 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] send_events_interval = 2 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.281725 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] service_plugins = ['router', 'neutron.services.metering.metering_plugin.MeteringPlugin'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.282013 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] setproctitle = on {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.282294 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] state_path = /opt/stack/data/neutron {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.282576 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] syslog_log_facility = LOG_USER {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.282868 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] tcp_keepidle = 600 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.283186 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] transport_url = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.283488 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] use_eventlog = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.283764 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] use_journal = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.284035 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] use_json = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.284309 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] use_ssl = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.284580 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] use_stderr = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.284864 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] use_syslog = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.285140 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] vlan_transparent = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.285416 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] watch_log_file = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.285695 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] wsgi_default_pool_size = 100 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.285968 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] wsgi_keep_alive = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.286251 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:13:59.286549 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_concurrency.disable_process_locking = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.286843 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_concurrency.lock_path = /opt/stack/data/neutron/lock {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.287142 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] profiler.connection_string = messaging:// {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.287452 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] profiler.enabled = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.287747 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] profiler.es_doc_type = notification {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.288027 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] profiler.es_scroll_size = 10000 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.288319 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] profiler.es_scroll_time = 2m {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.288599 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] profiler.filter_error_trace = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.288888 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] profiler.hmac_keys = SECRET_KEY {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.289175 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] profiler.sentinel_service_name = mymaster {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.289466 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] profiler.socket_timeout = 0.1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.289747 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] profiler.trace_sqlalchemy = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.290037 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_policy.enforce_new_defaults = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.290317 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_policy.enforce_scope = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.290612 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_policy.policy_default_rule = default {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.290916 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.291235 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_policy.policy_file = /etc/neutron/policy.json {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.291566 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.291847 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.292123 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.292398 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.292694 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.292995 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] nova.auth_section = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.293290 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] nova.auth_type = password {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.293584 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] nova.cafile = /opt/stack/data/ca-bundle.pem {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.293862 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] nova.certfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.294144 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] nova.collect_timing = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.294436 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] nova.endpoint_type = public {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.294717 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] nova.insecure = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.294992 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] nova.keyfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.295299 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] nova.live_migration_events = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.295598 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] nova.region_name = RegionOne {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.295882 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] nova.split_loggers = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.296158 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] nova.timeout = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.296439 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] placement.auth_section = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.296720 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] placement.auth_type = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.296993 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] placement.cafile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.297268 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] placement.certfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.297549 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] placement.collect_timing = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.297841 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] placement.endpoint_type = public {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.298123 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] placement.insecure = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.298396 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] placement.keyfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.298697 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] placement.region_name = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.298980 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] placement.split_loggers = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.299273 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] placement.timeout = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.299562 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.auth_section = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.299840 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.auth_type = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.300114 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.cafile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.300389 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.certfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.300668 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.collect_timing = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.300942 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.connect_retries = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.301215 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.connect_retry_delay = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.301495 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.enable_notifications = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.301771 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.endpoint_override = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.302053 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.insecure = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.302328 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.interface = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.302601 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.keyfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.302876 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.max_version = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.303147 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.min_version = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.303444 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.region_name = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.303719 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.service_name = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.303992 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.service_type = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.304268 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.split_loggers = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.304539 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.status_code_retries = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.304814 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.status_code_retry_delay = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.305087 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.timeout = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.305359 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.valid_interfaces = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.305637 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ironic.version = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.305926 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] service_providers.service_provider = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.306227 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.306507 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.306786 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.307064 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.307371 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.307654 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.307934 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.308212 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.308491 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.308765 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.309055 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.309338 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.309626 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.309912 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.310189 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.310483 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.310759 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.311040 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.311343 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.311622 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.311913 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.312197 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.312491 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.312784 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.313077 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.313369 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl_version = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.313663 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_notifications.driver = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.313946 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_notifications.retry = -1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.314310 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.314595 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] oslo_messaging_notifications.transport_url = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:13:59.314814 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.common.config [-] ******************************************************************************** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 20 04:13:59.315053 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo.service.wsgi [-] Loading app neutron from /etc/neutron/api-paste.ini {{(pid=91536) load_app /usr/local/lib/python3.8/dist-packages/oslo_service/wsgi.py:349}} May 20 04:13:59.659266 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" acquired by "neutron_lib.plugins.directory._create_plugin_directory" :: waited 0.000s {{(pid=91536) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:13:59.659510 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" released by "neutron_lib.plugins.directory._create_plugin_directory" :: held 0.001s {{(pid=91536) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:13:59.659808 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_concurrency.lockutils [-] Lock "manager" acquired by "neutron.manager.NeutronManager._create_instance" :: waited 0.000s {{(pid=91536) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:13:59.660133 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.manager [-] Loading core plugin: ml2 May 20 04:13:59.960245 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:13:59.960442 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:13:59.960710 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:13:59.961929 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:13:59.962201 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:13:59.962463 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.080078 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.080341 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.080620 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.093738 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.094016 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.094600 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.122694 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.122963 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.123262 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.160709 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.160988 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.161254 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.389393 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent after_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.390064 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent after_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.390064 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: segment precommit_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.390224 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: network precommit_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.390924 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: segment before_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.512509 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network before_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.512809 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent before_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.513084 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network before_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.513359 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.513628 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.513938 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment precommit_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.514198 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment precommit_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.514453 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment after_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.514706 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment after_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.514985 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network after_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.515240 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network precommit_delete 0 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.515518 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port provisioning_complete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.515780 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.516039 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy after_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.516302 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > agent after_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.516557 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet after_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.516817 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet precommit_delete 0 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.517341 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.517611 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.517902 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment_host_mapping after_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.518254 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.518513 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.518766 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.519051 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.519332 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.519585 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.519969 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: network configured {{(pid=91536) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 20 04:14:00.520209 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: port configured {{(pid=91536) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 20 04:14:00.520453 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnet configured {{(pid=91536) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 20 04:14:00.520690 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnetpool configured {{(pid=91536) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 20 04:14:00.520925 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group configured {{(pid=91536) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 20 04:14:00.521160 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group_rule configured {{(pid=91536) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 20 04:14:00.521526 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['local', 'flat', 'vlan', 'gre', 'vxlan', 'geneve'] May 20 04:14:00.524658 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.drivers.type_flat [-] Allowable flat physical_network names: ['physnet1'] May 20 04:14:00.528601 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete May 20 04:14:00.532427 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict([('physnet1', [(800, 849)])]) May 20 04:14:00.534343 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'geneve', 'gre', 'local', 'vlan', 'vxlan'] May 20 04:14:00.534807 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.managers [-] Registered types: dict_keys(['flat', 'geneve', 'gre', 'local', 'vlan', 'vxlan']) May 20 04:14:00.535176 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.managers [-] Tenant network_types: ['vlan'] May 20 04:14:00.535591 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.managers [-] Configured extension driver names: ['port_security'] May 20 04:14:00.538679 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: ['port_security'] May 20 04:14:00.538939 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.managers [-] Registered extension drivers: ['port_security'] May 20 04:14:00.539317 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['openvswitch', 'hyperv'] May 20 04:14:00.568086 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.drivers.openvswitch.mech_driver.mech_openvswitch [-] Open vSwitch agent's supported_vnic_types: ['normal', 'direct', 'smart-nic', 'vdpa'] May 20 04:14:00.568541 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > qos_plugin after_init 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.568846 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.services.qos.drivers.openvswitch.driver [-] Open vSwitch QoS driver registered {{(pid=91536) register /opt/stack/neutron/neutron/services/qos/drivers/openvswitch/driver.py:82}} May 20 04:14:00.569215 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > logging-plugin after_init 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.570576 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.services.logapi.drivers.openvswitch.driver [-] Open vSwitch logging driver registered {{(pid=91536) register /opt/stack/neutron/neutron/services/logapi/drivers/openvswitch/driver.py:69}} May 20 04:14:00.570858 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['openvswitch', 'hyperv'] May 20 04:14:00.571115 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['openvswitch', 'hyperv'] May 20 04:14:00.574235 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > floatingip before_response 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.574513 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_response 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:00.653635 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'flat' May 20 04:14:00.653795 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.drivers.type_flat [-] ML2 FlatTypeDriver initialization complete May 20 04:14:00.654035 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'geneve' May 20 04:14:00.654406 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.drivers.type_tunnel [-] geneve ID ranges: [(1, 1000)] May 20 04:14:00.688976 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_db.sqlalchemy.engines [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91536) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 20 04:14:01.023053 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.managers [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Initializing driver for type 'gre' May 20 04:14:01.023518 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.drivers.type_tunnel [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] gre ID ranges: [(1, 1000)] May 20 04:14:01.085324 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.managers [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Initializing driver for type 'local' May 20 04:14:01.085682 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.managers [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Initializing driver for type 'vlan' May 20 04:14:01.098798 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.drivers.type_vlan [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] VlanTypeDriver initialization complete May 20 04:14:01.099139 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.managers [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Initializing driver for type 'vxlan' May 20 04:14:01.099660 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.drivers.type_tunnel [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] vxlan ID ranges: [(1, 1000)] May 20 04:14:01.152528 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.managers [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Initializing extension driver 'port_security' May 20 04:14:01.152958 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.extensions.port_security [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] PortSecurityExtensionDriver initialization complete May 20 04:14:01.153322 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.managers [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Initializing mechanism driver 'openvswitch' May 20 04:14:01.153654 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.managers [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Initializing mechanism driver 'hyperv' May 20 04:14:01.160277 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.plugins.ml2.plugin [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] neutron.plugins.ml2.plugin.Ml2Plugin method _start_rpc_notifiers called with arguments () {} {{(pid=91536) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:14:01.161063 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > port after_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.161429 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > port after_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.161804 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > port after_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.162181 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > subnet after_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.162526 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > subnet after_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.162863 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > subnet after_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.163248 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > network after_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.163593 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > network after_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.163924 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > network after_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.164284 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > security_group after_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.164621 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > security_group after_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.164951 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > security_group after_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.165313 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > security_group_rule after_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.165640 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > security_group_rule after_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.165951 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > security_group_rule after_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.166306 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > address_group after_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.166630 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > address_group after_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.166956 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > address_group after_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.167299 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.plugins.ml2.ovo_rpc [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] ML2 OVO RPC backend initialized. {{(pid=91536) __init__ /opt/stack/neutron/neutron/plugins/ml2/ovo_rpc.py:160}} May 20 04:14:01.167984 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router_interface after_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.168324 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router_interface after_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.168728 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > network before_response 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.169063 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > networks before_response 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.169397 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > port before_response 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.169725 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > ports before_response 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.170054 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > subnet before_response 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.170377 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > subnets before_response 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.170694 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > network after_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.171021 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > network after_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.171377 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > network after_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.171704 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > port after_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.172028 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > port after_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.172347 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > port after_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.172669 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > subnet after_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.172991 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > subnet after_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.173309 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > subnet after_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.174014 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.plugin [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Modular L2 Plugin initialization complete May 20 04:14:01.174338 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading services supported by the core plugin {{(pid=91536) _load_services_from_core_plugin /opt/stack/neutron/neutron/manager.py:168}} May 20 04:14:01.174652 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.plugins.ml2.managers [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Got port-security extension from driver 'port_security' May 20 04:14:01.175101 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.extensions.vlantransparent [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Disabled vlantransparent extension. May 20 04:14:01.175600 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading service plugins: ['router', 'neutron.services.metering.metering_plugin.MeteringPlugin', 'auto_allocate', 'tag', 'timestamp', 'network_ip_availability', 'flavors', 'revisions'] {{(pid=91536) _load_service_plugins /opt/stack/neutron/neutron/manager.py:194}} May 20 04:14:01.175902 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading Plugin: router May 20 04:14:01.212568 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router_interface before_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.212944 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router after_create 55549900 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.213310 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router before_create 55549900 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.213665 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router_interface before_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.214025 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router_interface after_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.214376 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router after_delete 55549900 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.214720 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > floatingip after_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.215075 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router after_create 55549900 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.215437 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router after_update 55549900 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.215783 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router after_update 55549900 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.216119 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router_gateway after_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.216465 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router after_update 55549900 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.216843 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router precommit_update 55549900 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.217200 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: port after_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.217530 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: port after_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.217857 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: subnet after_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.218187 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: subnetpool_address_scope after_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.218519 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > port precommit_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.218854 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router precommit_create 55549900 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.219623 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: port before_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.219963 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router precommit_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.220306 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router after_update 55549900 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.220690 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router precommit_delete 55549900 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.221096 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router precommit_create 55549900 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.221451 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router_interface after_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.221786 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router precommit_update 55549900 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.222151 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > network after_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.223011 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Tracking information for resource: router configured {{(pid=91536) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 20 04:14:01.223362 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Tracking information for resource: floatingip configured {{(pid=91536) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 20 04:14:01.228258 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.db.l3_agentschedulers_db [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Skipping period L3 agent status check because automatic router rescheduling is disabled. May 20 04:14:01.228823 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: port after_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.229166 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: port after_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.229487 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: port after_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.229837 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: port after_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.230592 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router before_create 55550100 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.230944 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router precommit_delete 55550100 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.231320 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router precommit_create 55550100 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.231661 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > router precommit_update 55550100 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.236083 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.services.provider_configuration [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Service providers = [] {{(pid=91536) parse_service_provider_opt /opt/stack/neutron/neutron/services/provider_configuration.py:171}} May 20 04:14:01.236539 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.db.servicetype_db [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Adding provider configuration for service L3_ROUTER_NAT {{(pid=91536) add_provider_configuration /opt/stack/neutron/neutron/db/servicetype_db.py:42}} May 20 04:14:01.239392 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.services.service_base [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded 'neutron.services.l3_router.service_providers.dvrha.DvrHaDriver' provider for service L3_ROUTER_NAT {{(pid=91536) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 20 04:14:01.239735 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.services.service_base [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded 'neutron.services.l3_router.service_providers.dvr.DvrDriver' provider for service L3_ROUTER_NAT {{(pid=91536) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 20 04:14:01.240065 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.services.service_base [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded 'neutron.services.l3_router.service_providers.ha.HaDriver' provider for service L3_ROUTER_NAT {{(pid=91536) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 20 04:14:01.240930 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.services.service_base [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded 'neutron.services.l3_router.service_providers.single_node.SingleNodeDriver' provider for service L3_ROUTER_NAT {{(pid=91536) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 20 04:14:01.241297 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.services.service_base [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Default provider is not specified for service type L3_ROUTER_NAT May 20 04:14:01.241818 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Successfully loaded L3_ROUTER_NAT plugin. Description: L3 Router Service Plugin for basic L3 forwarding between (L2) Neutron networks and access to external networks via a NAT gateway. {{(pid=91536) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 20 04:14:01.242127 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading Plugin: neutron.services.metering.metering_plugin.MeteringPlugin May 20 04:14:01.252769 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Successfully loaded METERING plugin. Description: METERING {{(pid=91536) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 20 04:14:01.253096 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading Plugin: auto_allocate May 20 04:14:01.258684 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: network precommit_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.259049 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: network precommit_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.259575 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Successfully loaded auto-allocated-topology plugin. Description: Auto Allocated Topology - aka get me a network. {{(pid=91536) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 20 04:14:01.259888 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading Plugin: tag May 20 04:14:01.266612 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Successfully loaded TAG plugin. Description: Tag support {{(pid=91536) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 20 04:14:01.266929 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading Plugin: timestamp May 20 04:14:01.270793 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Successfully loaded timestamp plugin. Description: Adds timestamps to Neutron resources with standard attributes {{(pid=91536) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 20 04:14:01.271116 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading Plugin: network_ip_availability May 20 04:14:01.276324 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > rbac-policy before_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.276623 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > rbac-policy before_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.276964 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > rbac-policy before_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.277671 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Successfully loaded network-ip-availability plugin. Description: Provides IP availability data for each network and subnet. {{(pid=91536) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 20 04:14:01.277980 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading Plugin: flavors May 20 04:14:01.282781 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Successfully loaded FLAVORS plugin. Description: Neutron Flavors and Service Profiles manager plugin {{(pid=91536) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 20 04:14:01.283173 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading Plugin: revisions May 20 04:14:01.287437 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Successfully loaded revision_plugin plugin. Description: Adds revision numbers to resources. {{(pid=91536) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 20 04:14:01.287818 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_concurrency.lockutils [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Lock "manager" released by "neutron.manager.NeutronManager._create_instance" :: held 1.628s {{(pid=91536) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:14:01.288530 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] get_extension_paths = OrderedDict([('/opt/stack/neutron/neutron/extensions', 1)]) {{(pid=91536) get_extensions_path /opt/stack/neutron/neutron/api/extensions.py:635}} May 20 04:14:01.288861 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Initializing extension manager. May 20 04:14:01.289355 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: __init__.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.289683 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: __pycache__ {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.290005 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: _standard_attr_segment_lib.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.290324 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: address_group.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.291061 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Address group" alias="address-group" description="Support address group" updated="2020-07-15T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.291640 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: address-group May 20 04:14:01.291950 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: address_scope.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.292640 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Address scope" alias="address-scope" description="Address scopes extension." updated="2015-07-26T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.293137 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: address-scope May 20 04:14:01.293440 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: admin_state_down_before_update.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.294322 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Enforce Router's Admin State Down Before Update Extension" alias="router-admin-state-down-before-update" description="Ensure that the admin state of a router is DOWN (admin_state_up=False) before updating the distributed attribute" updated="2019-07-02T15:56:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.294819 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: router-admin-state-down-before-update May 20 04:14:01.295133 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: agent.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.295828 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="agent" alias="agent" description="The agent management extension." updated="2013-02-03T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.296324 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: agent May 20 04:14:01.296633 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: agent_resources_synced.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.297506 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Agent's Resource View Synced to Placement" alias="agent-resources-synced" description="Stores success/failure of last sync to Placement" updated="2018-12-19T00:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.298004 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: agent-resources-synced May 20 04:14:01.298313 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: allowedaddresspairs.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.300188 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Allowed Address Pairs" alias="allowed-address-pairs" description="Provides allowed address pairs" updated="2013-07-23T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.300708 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: allowed-address-pairs May 20 04:14:01.301029 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: auto_allocated_topology.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.302076 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Auto Allocated Topology Services" alias="auto-allocated-topology" description="Auto Allocated Topology Services." updated="2016-01-01T00:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.302581 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: auto-allocated-topology May 20 04:14:01.302890 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: availability_zone.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.303588 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Availability Zone" alias="availability_zone" description="The availability zone extension." updated="2015-01-01T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.304085 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: availability_zone May 20 04:14:01.304395 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: availability_zone_filter.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.305266 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Availability Zone Filter Extension" alias="availability_zone_filter" description="Add filter parameters to AvailabilityZone resource" updated="2018-06-22T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.305763 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: availability_zone_filter May 20 04:14:01.306075 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: data_plane_status.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.306944 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Port data plane status extension" alias="data-plane-status" description="Expose status of underlying data plane" updated="2017-01-24T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.307466 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension data-plane-status not supported by any of loaded plugins May 20 04:14:01.307777 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: default_subnetpools.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.308654 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Default Subnetpools" alias="default-subnetpools" description="Provides ability to mark and use a subnetpool as the default." updated="2016-02-18T18:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.309151 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: default-subnetpools May 20 04:14:01.309464 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: dhcpagentscheduler.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.310239 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="DHCP Agent Scheduler" alias="dhcp_agent_scheduler" description="Schedule networks among dhcp agents" updated="2013-02-07T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.310741 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: dhcp_agent_scheduler May 20 04:14:01.311056 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: dns.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.311951 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="DNS Integration" alias="dns-integration" description="Provides integration with DNS." updated="2015-08-15T18:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.312445 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension dns-integration not supported by any of loaded plugins May 20 04:14:01.312758 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: dns_domain_ports.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.313630 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="dns_domain for ports" alias="dns-domain-ports" description="Allows the DNS domain to be specified for a network port." updated="2017-04-24T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.314141 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension dns-domain-ports not supported by any of loaded plugins May 20 04:14:01.314540 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: dvr.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.315195 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Distributed Virtual Router" alias="dvr" description="Enables configuration of Distributed Virtual Routers." updated="2014-06-1T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.315717 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: dvr May 20 04:14:01.316025 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: empty_string_filtering.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.316903 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Empty String Filtering Extension" alias="empty-string-filtering" description="Allow filtering by attributes with empty string value" updated="2018-05-01T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.317388 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: empty-string-filtering May 20 04:14:01.317697 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: expose_l3_conntrack_helper.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.318574 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Expose CT target rules for conntrack helper" alias="expose-l3-conntrack-helper" description="Expose allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.319059 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension expose-l3-conntrack-helper not supported by any of loaded plugins May 20 04:14:01.319400 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: expose_port_forwarding_in_fip.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.320277 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Expose Floating IP port forwardings" alias="expose-port-forwarding-in-fip" description="Expose port forwarding fields in floating IP response" updated="2018-06-28T16:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.320764 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension expose-port-forwarding-in-fip not supported by any of loaded plugins May 20 04:14:01.321072 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: external_net.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.321942 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Neutron external network" alias="external-net" description="Adds external network attribute to network resource." updated="2013-01-14T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.322439 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: external-net May 20 04:14:01.322747 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: extra_dhcp_opt.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.323645 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Neutron Extra DHCP options" alias="extra_dhcp_opt" description="Extra options configuration for DHCP. For example PXE boot options to DHCP clients can be specified (e.g. tftp-server, server-ip-address, bootfile-name)" updated="2013-03-17T12:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.324145 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: extra_dhcp_opt May 20 04:14:01.324452 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: extraroute.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.325322 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Neutron Extra Route" alias="extraroute" description="Extra routes configuration for L3 router" updated="2013-02-01T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.325825 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: extraroute May 20 04:14:01.326138 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: extraroute_atomic.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.327135 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Atomically add/remove extra routes" alias="extraroute-atomic" description="Edit extra routes of a router on server side by atomically adding/removing extra routes" updated="2019-07-10T00:00:00+00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.327672 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: extraroute-atomic May 20 04:14:01.327985 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: filter_validation.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.328623 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Filter parameters validation" alias="filter-validation" description="Provides validation on filter parameters." updated="2018-07-04T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.329121 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: filter-validation May 20 04:14:01.329427 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: fip_pf_description.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.330652 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Floating IP Port Forwarding new attribute description" alias="floating-ip-port-forwarding-description" description="Add a description field to Port Forwarding rules" updated="2019-11-01T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.331151 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension floating-ip-port-forwarding-description not supported by any of loaded plugins May 20 04:14:01.331486 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: fip_port_details.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.332365 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Floating IP Port Details Extension" alias="fip-port-details" description="Add port_details attribute to Floating IP resource" updated="2018-04-09T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.332865 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: fip-port-details May 20 04:14:01.333175 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: flavors.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.334404 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Neutron Service Flavors" alias="flavors" description="Flavor specification for Neutron advanced services." updated="2015-09-17T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.334903 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: flavors May 20 04:14:01.335247 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: floating_ip_port_forwarding.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.336863 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Floating IP Port Forwarding" alias="floating-ip-port-forwarding" description="Allow port forwarding from floating IP port to an internal port" updated="2018-01-19T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.337372 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension floating-ip-port-forwarding not supported by any of loaded plugins May 20 04:14:01.337687 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: floatingip_pools.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.338364 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Floating IP Pools Extension" alias="floatingip-pools" description="Provides a floating IP pools API." updated="2018-03-21T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.338859 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: floatingip-pools May 20 04:14:01.339172 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: ip_allocation.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.340072 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="IP Allocation" alias="ip_allocation" description="IP allocation extension." updated="2016-06-10T23:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.340572 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension ip_allocation not supported by any of loaded plugins May 20 04:14:01.340886 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: ip_substring_port_filtering.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.341770 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="IP address substring filtering" alias="ip-substring-filtering" description="Provides IP address substring filtering when listing ports" updated="2017-11-28T09:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.342270 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: ip-substring-filtering May 20 04:14:01.342574 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: l2_adjacency.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.343496 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="L2 Adjacency" alias="l2_adjacency" description="Display L2 Adjacency for Neutron Networks." updated="2016-04-12T16:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.343994 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension l2_adjacency not supported by any of loaded plugins May 20 04:14:01.344632 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: l3.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.345656 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Neutron L3 Router" alias="router" description="Router abstraction for basic L3 forwarding between L2 Neutron networks and access to external networks via a NAT gateway." updated="2012-07-20T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.346227 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: router May 20 04:14:01.346543 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: l3_conntrack_helper.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.348760 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="L3 Conntrack helper" alias="l3-conntrack-helper" description="Allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.349284 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension l3-conntrack-helper not supported by any of loaded plugins May 20 04:14:01.349604 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: l3_ext_gw_mode.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.350516 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Neutron L3 Configurable external gateway mode" alias="ext-gw-mode" description="Extension of the router abstraction for specifying whether SNAT should occur on the external gateway" updated="2013-03-28T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.351022 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: ext-gw-mode May 20 04:14:01.351381 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: l3_ext_ha_mode.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.352261 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="HA Router extension" alias="l3-ha" description="Adds HA capability to routers." updated="2014-04-26T00:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.352762 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: l3-ha May 20 04:14:01.353078 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: l3_flavors.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.353949 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Router Flavor Extension" alias="l3-flavors" description="Flavor support for routers." updated="2016-05-17T00:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.354447 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: l3-flavors May 20 04:14:01.354761 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: l3_port_ip_change_not_allowed.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.355673 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Prevent L3 router ports IP address change extension" alias="l3-port-ip-change-not-allowed" description="Prevent change of IP address for some L3 router ports" updated="2018-10-09T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.356174 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: l3-port-ip-change-not-allowed May 20 04:14:01.356485 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: l3agentscheduler.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.357482 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="L3 Agent Scheduler" alias="l3_agent_scheduler" description="Schedule routers among l3 agents" updated="2013-02-07T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.357985 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: l3_agent_scheduler May 20 04:14:01.358303 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: logging.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.359780 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Logging API Extension" alias="logging" description="Provides a logging API for resources such as security group and firewall." updated="2017-01-01T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.360299 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension logging not supported by any of loaded plugins May 20 04:14:01.360616 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: metering.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.361532 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Neutron Metering" alias="metering" description="Neutron Metering extension." updated="2013-06-12T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.362098 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: metering May 20 04:14:01.362415 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: metering_source_and_destination_fields.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.363501 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Neutron Metering with source and destination filtering" alias="metering_source_and_destination_fields" description="Neutron Metering extension that enables the use of source and destination IP prefixes to create metering label rules." updated="2020-08-21T08:12:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.364009 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: metering_source_and_destination_fields May 20 04:14:01.364327 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: multiprovidernet.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.365350 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Multi Provider Network" alias="multi-provider" description="Expose mapping of virtual networks to multiple physical networks" updated="2013-06-27T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.365854 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: multi-provider May 20 04:14:01.366171 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: netmtu.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.367051 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Network MTU" alias="net-mtu" description="Provides MTU attribute for a network resource." updated="2015-03-25T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.367587 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: net-mtu May 20 04:14:01.367904 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: netmtu_writable.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.368790 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Network MTU (writable)" alias="net-mtu-writable" description="Provides a writable MTU attribute for a network resource." updated="2017-07-12T00:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.369295 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: net-mtu-writable May 20 04:14:01.369610 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: network_availability_zone.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.370264 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Network Availability Zone" alias="network_availability_zone" description="Availability zone support for network." updated="2015-01-01T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.370757 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: network_availability_zone May 20 04:14:01.371069 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: network_ip_availability.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.372149 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Network IP Availability" alias="network-ip-availability" description="Provides IP availability data for each network and subnet." updated="2015-09-24T00:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.372722 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: network-ip-availability May 20 04:14:01.373038 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: network_segment_range.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.373775 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Neutron Network Segment Range" alias="network-segment-range" description="Provides support for the network segment range management" updated="2018-11-29T00:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.374273 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension network-segment-range not supported by any of loaded plugins May 20 04:14:01.374583 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: pagination.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.375575 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Pagination support" alias="pagination" description="Extension that indicates that pagination is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.375899 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: pagination May 20 04:14:01.376210 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: port_device_profile.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.377219 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Port device profile" alias="port-device-profile" description="Expose the port device profile (Cyborg)" updated="2020-12-17T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.377722 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: port-device-profile May 20 04:14:01.378036 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: port_mac_address_regenerate.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.378917 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Neutron Port MAC address regenerate" alias="port-mac-address-regenerate" description="Network port MAC address regenerate" updated="2018-05-03T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.379478 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: port-mac-address-regenerate May 20 04:14:01.379793 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: port_numa_affinity_policy.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.380693 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Port NUMA affinity policy" alias="port-numa-affinity-policy" description="Expose the port NUMA affinity policy" updated="2020-07-08T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.381188 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: port-numa-affinity-policy May 20 04:14:01.381499 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: port_resource_request.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.382369 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Port Resource Request" alias="port-resource-request" description="Expose resource request to Port" updated="2018-05-08T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.382859 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension port-resource-request not supported by any of loaded plugins May 20 04:14:01.383169 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: portbindings.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.384076 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Port Binding" alias="binding" description="Expose port bindings of a virtual port to external application" updated="2014-02-03T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.384572 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: binding May 20 04:14:01.384884 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: portbindings_extended.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.386095 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Port Bindings Extended" alias="binding-extended" description="Expose port bindings of a virtual port to external application" updated="2017-07-17T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.386594 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: binding-extended May 20 04:14:01.386909 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: portsecurity.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.387817 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Port Security" alias="port-security" description="Provides port security" updated="2012-07-23T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.388314 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: port-security May 20 04:14:01.388628 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: project_id.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.389561 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="project_id field enabled" alias="project-id" description="Extension that indicates that project_id field is enabled." updated="2016-09-09T09:09:09-09:09" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.390095 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: project-id May 20 04:14:01.390184 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: providernet.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.391075 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Provider Network" alias="provider" description="Expose mapping of virtual networks to physical networks" updated="2012-09-07T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.391602 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: provider May 20 04:14:01.391916 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: qos.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.395551 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Quality of Service" alias="qos" description="The Quality of Service extension." updated="2015-06-08T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.396062 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension qos not supported by any of loaded plugins May 20 04:14:01.396386 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: qos_bw_limit_direction.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.397596 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Direction for QoS bandwidth limit rule" alias="qos-bw-limit-direction" description="Allow to configure QoS bandwidth limit rule with specific direction: ingress or egress" updated="2017-04-10T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.398101 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension qos-bw-limit-direction not supported by any of loaded plugins May 20 04:14:01.398418 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: qos_bw_minimum_ingress.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.399333 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Ingress direction for QoS minimum bandwidth rule" alias="qos-bw-minimum-ingress" description="Allow to configure QoS minumum bandwidth rule with ingress direction." updated="2018-07-09T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.399834 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension qos-bw-minimum-ingress not supported by any of loaded plugins May 20 04:14:01.400155 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: qos_default.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.401037 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="QoS default policy" alias="qos-default" description="Expose the QoS default policy per project" updated="2017-041-06T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.401542 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension qos-default not supported by any of loaded plugins May 20 04:14:01.401857 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: qos_fip.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.402477 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Floating IP QoS" alias="qos-fip" description="The floating IP Quality of Service extension" updated="2017-07-20T00:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.402970 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension qos-fip not supported by any of loaded plugins May 20 04:14:01.403324 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: qos_gateway_ip.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.404212 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Router gateway IP QoS" alias="qos-gateway-ip" description="The Router gateway IP Quality of Service extension" updated="2018-02-24T00:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.404720 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension qos-gateway-ip not supported by any of loaded plugins May 20 04:14:01.405032 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: qos_port_network_policy.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.405905 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="QoS port network policy ID" alias="qos-port-network-policy" description="Adds a the QoS network ID to the port definition" updated="2019-11-01T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.406552 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension qos-port-network-policy not supported by any of loaded plugins May 20 04:14:01.406872 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: qos_rule_type_details.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.407788 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Details of QoS rule types" alias="qos-rule-type-details" description="Expose details about QoS rule types supported by loaded backend drivers" updated="2017-06-22T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.408338 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension qos-rule-type-details not supported by any of loaded plugins May 20 04:14:01.408658 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: qos_rules_alias.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.409673 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Quality of Service rules alias API" alias="qos-rules-alias" description="API to enable GET, PUT and DELETE operations on QoS policy rules without specifying policy ID" updated="2018-10-07T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.410183 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension qos-rules-alias not supported by any of loaded plugins May 20 04:14:01.410498 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: quotasv2.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.411526 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Quota management support" alias="quotas" description="Expose functions for quotas management per tenant" updated="2012-07-29T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.411848 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: quotas May 20 04:14:01.412159 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: quotasv2_detail.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.413734 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Quota details management support" alias="quota_details" description="Expose functions for quotas usage statistics per project" updated="2017-02-10T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.414065 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: quota_details May 20 04:14:01.414383 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: rbac.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.415442 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="RBAC Policies" alias="rbac-policies" description="Allows creation and modification of policies that control tenant access to resources." updated="2015-06-17T12:15:12-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.415960 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: rbac-policies May 20 04:14:01.416281 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: rbac_address_group.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.417174 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Add address_group type to RBAC" alias="rbac-address-group" description="Add address_group type to network RBAC" updated="2021-01-20T00:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.417680 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: rbac-address-group May 20 04:14:01.417993 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: rbac_address_scope.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.418874 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Add address_scope type to RBAC" alias="rbac-address-scope" description="Add address_scope type to RBAC" updated="2020-02-12T00:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.419413 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: rbac-address-scope May 20 04:14:01.419726 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: rbac_security_groups.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.420607 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Add security_group type to network RBAC" alias="rbac-security-groups" description="Add security_group type to network RBAC" updated="2019-02-14T00:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.421112 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: rbac-security-groups May 20 04:14:01.421424 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: rbac_subnetpool.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.422315 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Add subnetpool type to RBAC" alias="rbac-subnetpool" description="Add subnetpool type to RBAC" updated="2020-02-05T00:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.422817 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: rbac-subnetpool May 20 04:14:01.423126 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: revisionifmatch.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.424153 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="If-Match constraints based on revision_number" alias="revision-if-match" description="Extension indicating that If-Match based on revision_number is supported." updated="2016-12-11T00:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.424651 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: revision-if-match May 20 04:14:01.424963 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: revisions.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.426037 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Resource revision numbers" alias="standard-attr-revisions" description="This extension will display the revision number of neutron resources." updated="2016-04-11T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.426538 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: standard-attr-revisions May 20 04:14:01.426853 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: router_availability_zone.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.427534 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Router Availability Zone" alias="router_availability_zone" description="Availability zone support for router." updated="2015-01-01T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.428041 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: router_availability_zone May 20 04:14:01.428351 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: routerservicetype.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.429236 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Router Service Type" alias="router-service-type" description="Provides router service type" updated="2013-01-29T00:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.429734 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension router-service-type not supported by any of loaded plugins May 20 04:14:01.430051 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: security_groups_normalized_cidr.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.430929 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Normalized CIDR field for security group rules" alias="security-groups-normalized-cidr" description="Add new field with normalized remote_ip_prefix cidr in SG rule" updated="2020-07-28T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.431464 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: security-groups-normalized-cidr May 20 04:14:01.431779 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: security_groups_port_filtering.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.432655 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Port filtering on security groups" alias="port-security-groups-filtering" description="Provides security groups filtering when listing ports" updated="2018-01-09T09:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.433155 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: port-security-groups-filtering May 20 04:14:01.433467 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: security_groups_remote_address_group.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.434352 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Remote address group id field for security group rules" alias="security-groups-remote-address-group" description="Add new field of remote address group id in SG rules" updated="2020-08-25T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.434853 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: security-groups-remote-address-group May 20 04:14:01.435165 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: securitygroup.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.444763 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="security-group" alias="security-group" description="The security groups extension." updated="2012-10-05T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.445295 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: security-group May 20 04:14:01.445619 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: segment.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.446365 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Segment" alias="segment" description="Segments extension." updated="2016-02-24T17:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.446862 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension segment not supported by any of loaded plugins May 20 04:14:01.447176 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: segments_peer_subnet_host_routes.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.448110 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Segments peer-subnet host routes" alias="segments-peer-subnet-host-routes" description="Add host routes to subnets on a routed network (segments)" updated="2018-06-12T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.448609 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension segments-peer-subnet-host-routes not supported by any of loaded plugins May 20 04:14:01.448927 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: servicetype.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.449995 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Neutron Service Type Management" alias="service-type" description="API for retrieving service providers for Neutron advanced services" updated="2013-01-20T00:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.450508 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: service-type May 20 04:14:01.450822 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: sorting.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.451791 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Sorting support" alias="sorting" description="Extension that indicates that sorting is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.452113 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: sorting May 20 04:14:01.452411 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: standard_attr_segment.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.453298 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Standard Attribute Segment Extension" alias="standard-attr-segment" description="Add standard attributes to Segment resource" updated="2018-04-09T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.453815 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension standard-attr-segment not supported by any of loaded plugins May 20 04:14:01.454133 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: standardattrdescription.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.454768 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="standard-attr-description" alias="standard-attr-description" description="Extension to add descriptions to standard attributes" updated="2016-02-10T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.455399 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: standard-attr-description May 20 04:14:01.455715 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: stateful_security_group.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.456601 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Stateful security group" alias="stateful-security-group" description="Indicates if the security group is stateful or not" updated="2019-11-26T09:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.457111 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: stateful-security-group May 20 04:14:01.457428 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: stdattrs_common.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.457981 dv-cinder-782405-9-iscsi neutron-server[91536]: WARNING neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Did not find expected name "Stdattrs_common" in /opt/stack/neutron/neutron/extensions/stdattrs_common.py May 20 04:14:01.458290 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: subnet_dns_publish_fixed_ip.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.459272 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Subnet DNS publish fixed IP" alias="subnet-dns-publish-fixed-ip" description="Support choosing to publish DNS records for IPs from a subnet" updated="2019-05-24T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.459777 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension subnet-dns-publish-fixed-ip not supported by any of loaded plugins May 20 04:14:01.460095 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: subnet_onboard.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.461125 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Subnet Onboard" alias="subnet_onboard" description="Provides support for onboarding subnets into subnet pools" updated="2018-12-18T09:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.461633 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: subnet_onboard May 20 04:14:01.461944 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: subnet_segmentid_writable.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.462910 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Subnet SegmentID (writable)" alias="subnet-segmentid-writable" description="Provides a writable segment_id attribute for a subnet resource." updated="2018-03-12T00:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.463443 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension subnet-segmentid-writable not supported by any of loaded plugins May 20 04:14:01.463752 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: subnet_service_types.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.464631 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Subnet service types" alias="subnet-service-types" description="Provides ability to set the subnet service_types field" updated="2016-03-15T18:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.465142 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: subnet-service-types May 20 04:14:01.465455 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: subnetallocation.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.466500 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Subnet Allocation" alias="subnet_allocation" description="Enables allocation of subnets from a subnet pool" updated="2015-03-30T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.467001 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: subnet_allocation May 20 04:14:01.467337 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: subnetpool_prefix_ops.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.467972 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Subnet Pool Prefix Operations" alias="subnetpool-prefix-ops" description="Provides support for adjusting the prefix list of subnet pools" updated="2019-02-08T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.468472 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: subnetpool-prefix-ops May 20 04:14:01.468783 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: tag_ports_during_bulk_creation.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.469993 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Tag Ports During Bulk Creation" alias="tag-ports-during-bulk-creation" description="Allow to tag ports during bulk creation" updated="2019-12-29T19:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.470609 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension tag-ports-during-bulk-creation not supported by any of loaded plugins May 20 04:14:01.470982 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: tagging.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.472042 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Tag support for resources with standard attribute: port, subnet, subnetpool, network, security_group, router, floatingip, policy, trunk, network_segment_range" alias="standard-attr-tag" description="Enables to set tag on resources with standard attribute." updated="2017-01-01T00:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.472565 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: standard-attr-tag May 20 04:14:01.472881 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: timestamp.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.474025 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Resource timestamps" alias="standard-attr-timestamp" description="Adds created_at and updated_at fields to all Neutron resources that have Neutron standard attributes." updated="2016-09-12T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.474534 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loaded extension: standard-attr-timestamp May 20 04:14:01.474854 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: trunk.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.475886 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Trunk Extension" alias="trunk" description="Provides support for trunk ports" updated="2016-01-01T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.476391 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension trunk not supported by any of loaded plugins May 20 04:14:01.476706 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: trunk_details.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.477603 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Trunk port details" alias="trunk-details" description="Expose trunk port details" updated="2016-01-01T10:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.478100 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension trunk-details not supported by any of loaded plugins May 20 04:14:01.478414 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: uplink_status_propagation.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.479339 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Uplink status propagation extension" alias="uplink-status-propagation" description="Provides ability to propagate port uplink status." updated="2018-05-31T18:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.479853 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension uplink-status-propagation not supported by any of loaded plugins May 20 04:14:01.480171 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Loading extension file: vlantransparent.py {{(pid=91536) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 20 04:14:01.480814 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Ext name="Vlantransparent" alias="vlan-transparent" description="Provides Vlan Transparent Networks" updated="2015-03-23T09:00:00-00:00" {{(pid=91536) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 20 04:14:01.481319 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extension vlan-transparent not supported by any of loaded plugins May 20 04:14:01.483771 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Creating instance of TrackedResource for resource:network May 20 04:14:01.484464 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Creating instance of TrackedResource for resource:subnet May 20 04:14:01.485213 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Creating instance of TrackedResource for resource:subnetpool May 20 04:14:01.485807 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Creating instance of TrackedResource for resource:port May 20 04:14:01.487163 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.v2.resource_helper [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=91536) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 20 04:14:01.487632 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.v2.resource_helper [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Service FLAVORS assigned prefix: {{(pid=91536) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 20 04:14:01.488231 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.v2.resource_helper [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=91536) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 20 04:14:01.488655 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.v2.resource_helper [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=91536) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 20 04:14:01.488975 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Creating instance of TrackedResource for resource:router May 20 04:14:01.489621 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Creating instance of TrackedResource for resource:floatingip May 20 04:14:01.490311 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.v2.resource_helper [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Service METERING assigned prefix: /metering {{(pid=91536) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 20 04:14:01.491769 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > rbac-policy before_create 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.492127 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > rbac-policy before_update 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.492470 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Subscribe: > rbac-policy before_delete 55550000 {{(pid=91536) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:14:01.493555 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Creating instance of CountableResource for resource:rbac_policy May 20 04:14:01.494001 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Creating instance of TrackedResource for resource:security_group May 20 04:14:01.494639 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Creating instance of TrackedResource for resource:security_group_rule May 20 04:14:01.495391 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.v2.resource_helper [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Service CORE assigned prefix: {{(pid=91536) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 20 04:14:01.495717 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Creating instance of TrackedResource for resource:subnetpool May 20 04:14:01.496043 dv-cinder-782405-9-iscsi neutron-server[91536]: WARNING neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] subnetpool is already registered May 20 04:14:01.496625 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.v2.resource_helper [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Service CORE assigned prefix: {{(pid=91536) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 20 04:14:01.496942 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Creating instance of TrackedResource for resource:subnetpool May 20 04:14:01.497255 dv-cinder-782405-9-iscsi neutron-server[91536]: WARNING neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] subnetpool is already registered May 20 04:14:01.500757 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.v2.resource_helper [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=91536) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 20 04:14:01.501178 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.v2.resource_helper [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Service FLAVORS assigned prefix: {{(pid=91536) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 20 04:14:01.501861 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.v2.resource_helper [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=91536) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 20 04:14:01.502325 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.v2.resource_helper [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=91536) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 20 04:14:01.502645 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Creating instance of TrackedResource for resource:router May 20 04:14:01.502962 dv-cinder-782405-9-iscsi neutron-server[91536]: WARNING neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] router is already registered May 20 04:14:01.503865 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Creating instance of TrackedResource for resource:floatingip May 20 04:14:01.504190 dv-cinder-782405-9-iscsi neutron-server[91536]: WARNING neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] floatingip is already registered May 20 04:14:01.504800 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.v2.resource_helper [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Service METERING assigned prefix: /metering {{(pid=91536) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 20 04:14:01.505458 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Creating instance of CountableResource for resource:rbac_policy May 20 04:14:01.505773 dv-cinder-782405-9-iscsi neutron-server[91536]: WARNING neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] rbac_policy is already registered May 20 04:14:01.506313 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Creating instance of TrackedResource for resource:security_group May 20 04:14:01.506632 dv-cinder-782405-9-iscsi neutron-server[91536]: WARNING neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] security_group is already registered May 20 04:14:01.507234 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Creating instance of TrackedResource for resource:security_group_rule May 20 04:14:01.507632 dv-cinder-782405-9-iscsi neutron-server[91536]: WARNING neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] security_group_rule is already registered May 20 04:14:01.508262 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.v2.resource_helper [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Service CORE assigned prefix: {{(pid=91536) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 20 04:14:01.508578 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Creating instance of TrackedResource for resource:subnetpool May 20 04:14:01.508897 dv-cinder-782405-9-iscsi neutron-server[91536]: WARNING neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] subnetpool is already registered May 20 04:14:01.509484 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.v2.resource_helper [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Service CORE assigned prefix: {{(pid=91536) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 20 04:14:01.509804 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Creating instance of TrackedResource for resource:subnetpool May 20 04:14:01.510134 dv-cinder-782405-9-iscsi neutron-server[91536]: WARNING neutron.quota.resource_registry [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] subnetpool is already registered May 20 04:14:01.510768 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: extensions {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.512219 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: address-groups {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.513843 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: address-scopes {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.515032 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: agents {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.516223 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: auto-allocated-topology {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.517549 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: availability_zones {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.518883 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: dhcp-networks {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.520299 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: dhcp-agents {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.521697 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: routers {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.523485 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: flavors {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.524637 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: service_profiles {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.525793 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: next_providers {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.527367 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: service_profiles {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.528735 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: floatingip-pools {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.530082 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: routers {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.531901 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: floatingips {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.533142 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: l3-routers {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.534586 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: l3-agents {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.536002 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: metering-labels {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.537335 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: metering-label-rules {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.538600 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: network-ip-availabilities {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.539882 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: bindings {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.541517 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: quotas {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.542905 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: quotas {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.544352 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: rbac-policies {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.545584 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: security-groups {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.546763 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: security-group-rules {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.547995 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: service-providers {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.549350 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: subnetpools {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.550737 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: subnetpools {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.552207 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: tags {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.553856 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: tags {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.555652 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: tags {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.557389 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: tags {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.560162 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: tags {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.561979 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: tags {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.563765 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: tags {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.677551 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: tags {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.679342 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: tags {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.681148 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron.api.extensions [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Extended resource: tags {{(pid=91536) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 20 04:14:01.686952 dv-cinder-782405-9-iscsi neutron-server[91536]: WARNING keystonemiddleware.auth_token [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 20 04:14:01.691821 dv-cinder-782405-9-iscsi neutron-server[91536]: WARNING keystonemiddleware.auth_token [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 20 04:14:01.699885 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_concurrency.lockutils [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Acquired lock "singleton_lock" {{(pid=91536) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:266}} May 20 04:14:01.700383 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_concurrency.lockutils [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Releasing lock "singleton_lock" {{(pid=91536) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:282}} May 20 04:14:01.701015 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO oslo_service.service [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Starting 2 workers May 20 04:14:01.705732 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Started child 91643 {{(pid=91536) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:576}} May 20 04:14:01.709840 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Started child 91644 {{(pid=91536) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:576}} May 20 04:14:01.710809 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [-] Notify callbacks [] for process, after_init {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:01.711597 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO neutron.service [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Neutron service started, listening on 0.0.0.0:9696 May 20 04:14:01.711949 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Notify callbacks [] for process, before_spawn {{(pid=91536) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:01.712322 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [-] (91643) wsgi starting up on http://0.0.0.0:9696 May 20 04:14:01.714281 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO oslo_service.service [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Starting 1 workers May 20 04:14:01.714474 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [-] Notify callbacks [] for process, after_init {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:01.715707 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [-] (91644) wsgi starting up on http://0.0.0.0:9696 May 20 04:14:01.719120 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Started child 91645 {{(pid=91536) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:576}} May 20 04:14:01.719956 dv-cinder-782405-9-iscsi neutron-server[91536]: INFO oslo_service.service [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Starting 1 workers May 20 04:14:01.722818 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.service [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] calling RpcWorker wait() {{(pid=91645) _wait /opt/stack/neutron/neutron/service.py:128}} May 20 04:14:01.723432 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.service [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] returning from RpcWorker wait() {{(pid=91645) _wait /opt/stack/neutron/neutron/service.py:135}} May 20 04:14:01.724216 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Started child 91646 {{(pid=91536) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:576}} May 20 04:14:01.724500 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [-] Notify callbacks [] for process, after_init {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:01.724955 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG neutron_lib.callbacks.manager [None req-1b6ae1ea-3643-4860-8c64-e0946aba5193 None None] Notify callbacks [] for process, after_spawn {{(pid=91536) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:01.725085 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [-] neutron.plugins.ml2.plugin.Ml2Plugin method start_rpc_listeners called with arguments () {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:14:01.726684 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] Full set of CONF: {{(pid=91536) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:650}} May 20 04:14:01.726898 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ******************************************************************************** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 20 04:14:01.727056 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] Configuration options gathered from: {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 20 04:14:01.727279 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 20 04:14:01.727603 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 20 04:14:01.727779 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ================================================================================ {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 20 04:14:01.728269 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] agent_down_time = 75 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.728501 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] allow_automatic_dhcp_failover = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.728787 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] allow_automatic_l3agent_failover = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.728915 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [-] Notify callbacks [] for process, after_init {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:01.729382 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] allow_bulk = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.729382 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] allow_overlapping_ips = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.729871 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] allowed_conntrack_helpers = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.730099 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] api_extensions_path = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.730380 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] api_paste_config = api-paste.ini {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.730648 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] api_workers = 2 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.730871 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] auth_strategy = keystone {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.731030 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] backdoor_port = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.731175 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] backdoor_socket = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.731459 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] backlog = 4096 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.731687 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] base_mac = fa:16:3e:00:00:00 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.731992 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] bind_host = 0.0.0.0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.732250 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] bind_port = 9696 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.732480 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] client_socket_timeout = 900 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.732744 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] config_dir = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.732950 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] config_file = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.733174 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] config_source = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.733451 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] control_exchange = neutron {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.733672 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] core_plugin = ml2 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.733918 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] debug = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.734122 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] default_availability_zones = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.734584 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.734821 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] dhcp_agent_notification = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.735013 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] dhcp_agents_per_network = 1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.735240 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] dhcp_lease_duration = 86400 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.735544 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] dhcp_load_type = networks {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.735748 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] dns_domain = openstacklocal {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.736149 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] dvr_base_mac = fa:16:3f:00:00:00 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.736411 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] enable_dvr = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.736678 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] enable_new_agents = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.736903 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] enable_services_on_agents_with_admin_state_down = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.737162 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] enable_snat_by_default = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.737390 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] enable_traditional_dhcp = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.737606 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] external_dns_driver = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.737802 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] filter_validation = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.738078 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] global_physnet_mtu = 1500 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.738299 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] graceful_shutdown_timeout = 60 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.738786 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] host = dv-cinder-782405-9-iscsi {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.739013 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] host_dvr_for_dhcp = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.739240 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] http_retries = 3 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.739484 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] instance_format = [instance: %(uuid)s] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.739729 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.739957 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ipam_driver = internal {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.740195 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ipv6_pd_enabled = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.740446 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] l3_ha = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.740679 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] l3_ha_net_cidr = 169.254.192.0/18 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.740915 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] l3_ha_network_physical_name = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.741126 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] l3_ha_network_type = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.741370 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] log_config_append = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.741599 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.741809 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] log_dir = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.742039 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] log_file = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.742196 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] log_options = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.742405 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] log_rotate_interval = 1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.742684 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] log_rotate_interval_type = days {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.742941 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] log_rotation_type = none {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.743124 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.743318 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.743477 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.743705 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.743863 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.744099 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] max_allowed_address_pair = 10 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.744326 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] max_dns_nameservers = 5 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.744538 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] max_header_line = 16384 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.744752 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] max_l3_agents_per_router = 3 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.744980 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] max_logfile_count = 30 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.745208 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] max_logfile_size_mb = 200 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.745441 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] max_routes = 30 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.745685 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] max_subnet_host_routes = 20 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.745919 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] network_auto_schedule = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.746123 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] network_link_prefix = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.746381 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] network_scheduler_driver = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.746604 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] notify_nova_on_port_data_changes = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.746824 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] notify_nova_on_port_status_changes = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.747043 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] pagination_max_limit = -1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.747273 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] periodic_fuzzy_delay = 5 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.747483 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] periodic_interval = 40 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.747713 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] publish_errors = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.747928 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] rate_limit_burst = 0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.748184 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] rate_limit_except_level = CRITICAL {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.748377 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] rate_limit_interval = 0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.748624 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] retry_until_window = 30 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.748823 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] router_auto_schedule = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.749049 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] router_distributed = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.749296 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] router_scheduler_driver = neutron.scheduler.l3_agent_scheduler.LeastRoutersScheduler {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.749524 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] rpc_response_max_timeout = 600 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.749763 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] rpc_response_timeout = 60 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.750012 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] rpc_state_report_workers = 0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.751982 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] rpc_workers = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.752249 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] send_events_interval = 2 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.752549 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] service_plugins = ['router', 'neutron.services.metering.metering_plugin.MeteringPlugin'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.752801 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] setproctitle = on {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.753075 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] state_path = /opt/stack/data/neutron {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.753338 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] syslog_log_facility = LOG_USER {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.753761 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] tcp_keepidle = 600 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.753929 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.services.l3_router.l3_router_plugin [-] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method start_rpc_listeners called with arguments () {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:14:01.754049 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] transport_url = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.754049 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] use_eventlog = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.754262 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] use_journal = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.754484 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] use_json = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.754704 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] use_ssl = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.754922 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] use_stderr = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.755154 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] use_syslog = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.755408 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] vlan_transparent = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.755827 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] watch_log_file = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.756719 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] wsgi_default_pool_size = 100 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.757254 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] wsgi_keep_alive = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.758180 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.758487 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_concurrency.disable_process_locking = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.758777 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_concurrency.lock_path = /opt/stack/data/neutron/lock {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.761251 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] profiler.connection_string = messaging:// {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.762030 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] profiler.enabled = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.762030 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] profiler.es_doc_type = notification {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.762030 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] profiler.es_scroll_size = 10000 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.762475 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] profiler.es_scroll_time = 2m {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.762475 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] profiler.filter_error_trace = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.762847 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] profiler.hmac_keys = SECRET_KEY {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.762981 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] profiler.sentinel_service_name = mymaster {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.763295 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] profiler.socket_timeout = 0.1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.763538 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] profiler.trace_sqlalchemy = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.763899 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_policy.enforce_new_defaults = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.764482 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_policy.enforce_scope = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.764482 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_policy.policy_default_rule = default {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.764723 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.764889 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_policy.policy_file = /etc/neutron/policy.json {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.765149 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.765384 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.765603 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.766111 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.766111 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.766559 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.auth_section = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.766688 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.auth_type = password {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.767083 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.auth_url = http://10.222.0.39/identity {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.767083 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.cafile = /opt/stack/data/ca-bundle.pem {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.767614 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.certfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.767740 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.collect_timing = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.767882 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.default_domain_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.768054 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.default_domain_name = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.768292 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.domain_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.768520 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.domain_name = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.768787 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.endpoint_type = public {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.769014 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.insecure = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.769232 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.keyfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.769457 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.live_migration_events = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.769695 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.password = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.769911 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.project_domain_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.770282 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.project_domain_name = Default {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.770412 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.project_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.770631 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.project_name = service {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.770822 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.region_name = RegionOne {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.771164 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.split_loggers = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.771318 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.system_scope = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.771515 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.timeout = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.771748 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.trust_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.771956 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.user_domain_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.772461 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.user_domain_name = Default {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.772461 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.user_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.772703 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.username = nova {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.772907 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.auth_section = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.773129 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.auth_type = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.773546 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.cafile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.773546 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.certfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.773799 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.collect_timing = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.773966 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.endpoint_type = public {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.774184 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.insecure = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.774417 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.keyfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.774630 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.region_name = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.774848 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.split_loggers = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.775076 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.timeout = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.775384 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.auth_section = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.775593 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.auth_type = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.775828 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.cafile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.776042 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.certfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.776484 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.collect_timing = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.776484 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.connect_retries = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.776729 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.connect_retry_delay = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.776942 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.enable_notifications = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.777295 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.endpoint_override = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.777509 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.insecure = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.777644 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.interface = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.777766 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.keyfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.777964 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.max_version = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.778195 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.min_version = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.778407 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.region_name = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.778623 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.service_name = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.778832 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.service_type = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.779048 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.split_loggers = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.779285 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.status_code_retries = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.779520 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.status_code_retry_delay = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.779736 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.timeout = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.779947 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.valid_interfaces = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.780170 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.version = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.780451 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] service_providers.service_provider = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.780763 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.781006 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.781265 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.781516 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.781754 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.781972 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.782221 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.782615 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.782733 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.782881 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.783151 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.783393 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.783646 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.783893 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.784128 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.784379 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.784602 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.784855 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.785078 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.785314 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.785567 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.785835 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.786083 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.786344 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.786608 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.786838 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_version = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.787120 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_notifications.driver = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.787380 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_notifications.retry = -1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.787627 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.787848 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_notifications.transport_url = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.788120 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.788415 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.default_quota = -1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.788742 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.quota_driver = neutron.db.quota.driver.DbQuotaDriver {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.788955 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.quota_floatingip = 50 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.789221 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.quota_network = 100 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.789813 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.quota_port = 500 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.789813 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.quota_rbac_policy = 10 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.790090 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.quota_router = 100 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.790258 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.quota_security_group = 100 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.790393 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.quota_security_group_rule = 100 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.790630 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.quota_subnet = 100 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.790844 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.track_quota_usage = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.791100 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_middleware.enable_proxy_headers_parsing = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.791378 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] healthcheck.backends = ['disable_by_file'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.791598 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] healthcheck.detailed = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.791830 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] healthcheck.disable_by_file_path = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.792058 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] healthcheck.path = /healthcheck {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.792375 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.admin_password = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.792605 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.792853 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.admin_token = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.793073 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.admin_user = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.793337 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.auth_admin_prefix = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.793565 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.793813 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.auth_port = 35357 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.794067 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.auth_protocol = https {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.794321 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.auth_section = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.794573 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.auth_type = password {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.794824 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.auth_uri = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.795074 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.auth_url = http://10.222.0.39/identity {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.795407 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.auth_version = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.795646 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.cache = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.795880 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.cafile = /opt/stack/data/ca-bundle.pem {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.796094 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.certfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.797021 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.default_domain_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.797021 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.default_domain_name = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.797021 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.delay_auth_decision = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.797021 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.domain_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.797725 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.domain_name = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.797725 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.797725 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.http_connect_timeout = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.798127 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.798127 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.identity_uri = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.799095 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.include_service_catalog = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.799095 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.insecure = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.799095 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.interface = public {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.799095 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.keyfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.799445 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.799445 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.799582 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.799790 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.800527 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.800527 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.800527 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.800749 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_use_advanced_pool = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.800906 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.801173 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.password = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.801386 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.project_domain_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.801631 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.project_domain_name = Default {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.801872 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.project_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.802104 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.project_name = service {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.802552 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.region_name = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.802552 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.803280 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.service_token_roles_required = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.803280 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.service_type = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.803280 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.system_scope = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.804004 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.token_cache_time = 300 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.804004 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.trust_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.804004 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.user_domain_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.804004 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.user_domain_name = Default {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.804544 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.user_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.804544 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.username = neutron {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.804821 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.805038 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.backend = dogpile.cache.null {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.805230 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.backend_argument = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.805471 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.config_prefix = cache.oslo {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.805729 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.debug_cache_backend = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.805941 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.enabled = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.806156 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.expiration_time = 600 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.806401 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.memcache_dead_retry = 300 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.806633 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.806845 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.807076 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.memcache_pool_maxsize = 10 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.807311 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.807574 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.807822 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.memcache_socket_timeout = 1.0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.808056 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.proxies = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.808273 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.tls_allowed_ciphers = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.808485 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.tls_cafile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.808711 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.tls_certfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.808963 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.tls_enabled = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.809126 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.tls_keyfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.809405 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep.capabilities = [21, 12, 1, 2, 19] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.809651 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep.group = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.809877 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep.helper_command = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.810121 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep.thread_pool_size = 4 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.810351 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep.user = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.810578 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.810787 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_dhcp_release.group = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.810995 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_dhcp_release.helper_command = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.811255 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_dhcp_release.thread_pool_size = 4 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.811419 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_dhcp_release.user = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.811638 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.811846 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.group = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.812052 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.helper_command = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.812267 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.thread_pool_size = 4 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.812474 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.user = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.812715 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_namespace.capabilities = [21] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.812922 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_namespace.group = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.813129 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_namespace.helper_command = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.813362 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_namespace.thread_pool_size = 4 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.813569 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_namespace.user = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.813789 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_conntrack.capabilities = [12] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.814014 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_conntrack.group = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.814248 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_conntrack.helper_command = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.814471 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_conntrack.thread_pool_size = 4 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.814679 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_conntrack.user = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.815072 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] AGENT.root_helper = sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.815314 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] AGENT.root_helper_daemon = sudo /usr/local/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.815462 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] AGENT.use_helper_for_ns_read = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.815732 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] SECURITYGROUP.enable_ipset = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.816107 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] SECURITYGROUP.enable_security_group = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.816256 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] SECURITYGROUP.firewall_driver = openvswitch {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.816441 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.816773 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2.extension_drivers = ['port_security'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.816993 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2.external_network_type = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.817248 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2.mechanism_drivers = ['openvswitch', 'hyperv'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.817466 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2.overlay_ip_version = 4 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.817683 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2.path_mtu = 0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.817899 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2.physical_network_mtus = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.818149 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2.tenant_network_types = ['vlan'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.818469 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2.type_drivers = ['local', 'flat', 'vlan', 'gre', 'vxlan', 'geneve'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.818747 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2_type_flat.flat_networks = ['physnet1'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.818990 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2_type_geneve.max_header_size = 30 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.819277 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2_type_geneve.vni_ranges = ['1:1000'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.819558 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2_type_gre.tunnel_id_ranges = ['1:1000'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.819831 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2_type_vlan.network_vlan_ranges = ['physnet1:800:849'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.820102 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2_type_vxlan.vni_ranges = ['1:1000'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.820335 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2_type_vxlan.vxlan_group = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.820622 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] OVS_DRIVER.vnic_type_prohibit_list = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.820956 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.backend = sqlalchemy {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.821232 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.connection = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.821479 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.connection_debug = 0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.821703 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.connection_parameters = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.821970 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.connection_recycle_time = 3600 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.822196 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.connection_trace = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.823324 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.db_inc_retry_interval = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.823324 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.db_max_retries = 20 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.823324 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.db_max_retry_interval = 10 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.823324 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.db_retry_interval = 1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.823324 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.max_overflow = 50 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.823654 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.max_pool_size = 50 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.823837 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.max_retries = 10 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.823997 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.mysql_enable_ndb = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.824220 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.824434 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.pool_timeout = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.824680 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.retry_interval = 10 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.824917 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.slave_connection = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.825155 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.sqlite_synchronous = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.825389 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.use_db_reconnect = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.825548 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ******************************************************************************** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 20 04:14:01.826103 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] Full set of CONF: {{(pid=91536) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:650}} May 20 04:14:01.826262 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ******************************************************************************** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 20 04:14:01.826413 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] Configuration options gathered from: {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 20 04:14:01.826598 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 20 04:14:01.826765 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 20 04:14:01.826914 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ================================================================================ {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 20 04:14:01.827118 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] agent_down_time = 75 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.827297 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] allow_automatic_dhcp_failover = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.827456 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] allow_automatic_l3agent_failover = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.827613 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] allow_bulk = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.827771 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] allow_overlapping_ips = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.827948 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] allowed_conntrack_helpers = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.828108 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] api_extensions_path = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.828264 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] api_paste_config = api-paste.ini {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.828420 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] api_workers = 2 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.828579 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] auth_strategy = keystone {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.828734 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] backdoor_port = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.828891 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] backdoor_socket = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.829046 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] backlog = 4096 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.829202 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] base_mac = fa:16:3e:00:00:00 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.829358 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] bind_host = 0.0.0.0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.829514 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] bind_port = 9696 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.829691 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] client_socket_timeout = 900 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.829878 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] config_dir = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.830040 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] config_file = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.830199 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] config_source = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.830354 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] control_exchange = neutron {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.830509 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] core_plugin = ml2 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.830670 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] debug = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.830826 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] default_availability_zones = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.831010 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.831169 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] dhcp_agent_notification = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.831352 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] dhcp_agents_per_network = 1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.831510 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] dhcp_lease_duration = 86400 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.831664 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] dhcp_load_type = networks {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.831821 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] dns_domain = openstacklocal {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.831977 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] dvr_base_mac = fa:16:3f:00:00:00 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.832133 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] enable_dvr = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.832289 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] enable_new_agents = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.832438 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] enable_services_on_agents_with_admin_state_down = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.832599 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] enable_snat_by_default = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.832754 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] enable_traditional_dhcp = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.833064 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] external_dns_driver = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.833359 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] filter_validation = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.833654 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] global_physnet_mtu = 1500 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.833945 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] graceful_shutdown_timeout = 60 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.834280 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] host = dv-cinder-782405-9-iscsi {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.834437 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] host_dvr_for_dhcp = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.834598 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] http_retries = 3 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.834754 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] instance_format = [instance: %(uuid)s] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.834902 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.835058 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ipam_driver = internal {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.835203 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ipv6_pd_enabled = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.835385 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] l3_ha = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.835544 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] l3_ha_net_cidr = 169.254.192.0/18 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.835702 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] l3_ha_network_physical_name = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.835856 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] l3_ha_network_type = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.836190 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] log_config_append = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.836519 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.836680 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] log_dir = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.836919 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] log_file = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.837024 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] log_options = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.837139 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] log_rotate_interval = 1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.837292 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] log_rotate_interval_type = days {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.837594 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] log_rotation_type = none {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.837773 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.837938 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.838094 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.838268 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.838396 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.838554 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] max_allowed_address_pair = 10 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.838712 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] max_dns_nameservers = 5 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.838870 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] max_header_line = 16384 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.839025 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] max_l3_agents_per_router = 3 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.839185 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] max_logfile_count = 30 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.839367 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] max_logfile_size_mb = 200 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.839525 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] max_routes = 30 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.839683 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] max_subnet_host_routes = 20 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.839839 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] network_auto_schedule = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.840000 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] network_link_prefix = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.840155 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] network_scheduler_driver = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.840310 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] notify_nova_on_port_data_changes = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.840471 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] notify_nova_on_port_status_changes = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.840629 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] pagination_max_limit = -1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.840791 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] periodic_fuzzy_delay = 5 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.840943 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] periodic_interval = 40 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.841089 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] publish_errors = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.841245 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] rate_limit_burst = 0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.841401 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] rate_limit_except_level = CRITICAL {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.841558 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] rate_limit_interval = 0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.841715 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] retry_until_window = 30 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.841871 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] router_auto_schedule = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.842517 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] router_distributed = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.842517 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] router_scheduler_driver = neutron.scheduler.l3_agent_scheduler.LeastRoutersScheduler {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.842517 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] rpc_response_max_timeout = 600 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.842517 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] rpc_response_timeout = 60 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.842809 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] rpc_state_report_workers = 0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.842809 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] rpc_workers = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.842928 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] send_events_interval = 2 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.843044 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] service_plugins = ['router', 'neutron.services.metering.metering_plugin.MeteringPlugin'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.843198 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] setproctitle = on {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.843376 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] state_path = /opt/stack/data/neutron {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.843537 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] syslog_log_facility = LOG_USER {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.843694 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] tcp_keepidle = 600 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.843851 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] transport_url = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.844008 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] use_eventlog = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.844470 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] use_journal = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.844634 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] use_json = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.844790 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] use_ssl = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.844945 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] use_stderr = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.845101 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] use_syslog = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.845256 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] vlan_transparent = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.845438 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] watch_log_file = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.845594 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] wsgi_default_pool_size = 100 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.845747 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] wsgi_keep_alive = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.845911 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 04:14:01.846089 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_concurrency.disable_process_locking = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.846252 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_concurrency.lock_path = /opt/stack/data/neutron/lock {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.846424 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] profiler.connection_string = messaging:// {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.846587 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] profiler.enabled = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.846746 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] profiler.es_doc_type = notification {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.846905 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] profiler.es_scroll_size = 10000 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.847066 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] profiler.es_scroll_time = 2m {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.847243 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] profiler.filter_error_trace = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.847413 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] profiler.hmac_keys = SECRET_KEY {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.847568 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] profiler.sentinel_service_name = mymaster {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.847724 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] profiler.socket_timeout = 0.1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.847884 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] profiler.trace_sqlalchemy = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.848055 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_policy.enforce_new_defaults = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.848214 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_policy.enforce_scope = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.848374 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_policy.policy_default_rule = default {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.848541 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.848700 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_policy.policy_file = /etc/neutron/policy.json {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.848863 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.849024 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.849183 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.849348 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.849502 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.849829 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.auth_section = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.849991 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.auth_type = password {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.850136 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.auth_url = http://10.222.0.39/identity {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.850388 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.cafile = /opt/stack/data/ca-bundle.pem {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.850536 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.certfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.850669 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.collect_timing = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.850812 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.default_domain_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.850902 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.default_domain_name = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.851044 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.domain_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.851239 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.domain_name = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.851407 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.endpoint_type = public {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.851630 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.insecure = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.851771 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.keyfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.851894 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.live_migration_events = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.852017 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.password = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.852139 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.project_domain_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.852303 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.project_domain_name = Default {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.852463 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.project_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.852623 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.project_name = service {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.852748 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.services.l3_router.l3_router_plugin [-] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method start_rpc_listeners called with arguments () {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:14:01.852894 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.region_name = RegionOne {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.853020 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.split_loggers = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.853143 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.system_scope = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.853282 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.timeout = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.853471 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.trust_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.853597 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.user_domain_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.853719 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.user_domain_name = Default {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.853841 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.user_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.853973 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] nova.username = nova {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.854118 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.auth_section = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.854277 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.auth_type = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.854437 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.cafile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.854598 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.certfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.854756 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.collect_timing = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.854916 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.endpoint_type = public {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.855075 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.insecure = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.855275 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.keyfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.855424 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.region_name = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.855550 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.split_loggers = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.855714 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] placement.timeout = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.855888 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.auth_section = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.856047 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.auth_type = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.856208 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.cafile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.856368 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.certfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.856528 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.collect_timing = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.856688 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.connect_retries = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.856848 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.connect_retry_delay = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.857006 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.enable_notifications = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.857167 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.endpoint_override = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.857326 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.insecure = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.857484 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.interface = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.857644 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.keyfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.857803 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.max_version = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.857963 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.min_version = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.858122 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.region_name = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.858282 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.service_name = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.858442 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.service_type = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.858601 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.split_loggers = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.858760 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.status_code_retries = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.858919 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.status_code_retry_delay = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.859079 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.timeout = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.859276 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.valid_interfaces = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.859440 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ironic.version = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.859610 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] service_providers.service_provider = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.859785 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.859944 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.860105 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.860265 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.860425 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.860590 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.860750 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.860912 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.861117 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.861278 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.861437 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.861595 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.861800 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.861925 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.862085 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.862205 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.862327 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.862490 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.862650 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.862809 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.862967 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.863128 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.863311 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.863474 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.863636 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.863794 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_version = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.863945 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_notifications.driver = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.865474 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_notifications.retry = -1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.865474 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.865474 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_messaging_notifications.transport_url = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.865474 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.865474 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.default_quota = -1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.865474 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.quota_driver = neutron.db.quota.driver.DbQuotaDriver {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.865474 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.quota_floatingip = 50 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.865474 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.quota_network = 100 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.866990 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.quota_port = 500 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.866990 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.quota_rbac_policy = 10 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.866990 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.quota_router = 100 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.866990 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.quota_security_group = 100 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.866990 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.quota_security_group_rule = 100 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.866990 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.quota_subnet = 100 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.866990 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] QUOTAS.track_quota_usage = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.866990 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] oslo_middleware.enable_proxy_headers_parsing = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.869330 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] healthcheck.backends = ['disable_by_file'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.869330 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] healthcheck.detailed = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.869330 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] healthcheck.disable_by_file_path = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.869330 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] healthcheck.path = /healthcheck {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.869330 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.admin_password = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.869330 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.869330 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.admin_token = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.869330 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.admin_user = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.870495 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.auth_admin_prefix = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.870495 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.870495 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.auth_port = 35357 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.870495 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.auth_protocol = https {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.870495 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.auth_section = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.870495 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.auth_type = password {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.870495 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.auth_uri = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.870495 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.auth_url = http://10.222.0.39/identity {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.871358 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.auth_version = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.871358 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.cache = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.871358 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.cafile = /opt/stack/data/ca-bundle.pem {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.871358 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.certfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.871358 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.default_domain_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.871358 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.default_domain_name = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.871358 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.delay_auth_decision = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.871358 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.domain_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.872224 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.domain_name = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.872224 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.872224 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.http_connect_timeout = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.872224 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.872224 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.identity_uri = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.872224 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.include_service_catalog = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.872224 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.insecure = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.872224 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.interface = public {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.872828 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.keyfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.872828 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.872828 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.872828 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.872828 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.872828 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.872828 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.872828 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.873372 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_use_advanced_pool = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.873372 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.873372 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.password = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.873372 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.project_domain_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.873372 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.project_domain_name = Default {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.873372 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.project_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.873372 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.project_name = service {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.874125 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.region_name = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.874125 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.874125 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.service_token_roles_required = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.874125 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.service_type = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.874125 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.system_scope = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.874674 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.token_cache_time = 300 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.874674 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.trust_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.874674 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.user_domain_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.874674 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.user_domain_name = Default {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.874960 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.user_id = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.874960 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.username = neutron {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.875132 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.875326 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.backend = dogpile.cache.null {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.875487 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.backend_argument = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.875646 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.config_prefix = cache.oslo {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.875805 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.debug_cache_backend = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.875963 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.enabled = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.876122 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.expiration_time = 600 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.876281 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.memcache_dead_retry = 300 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.876439 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.876619 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.876778 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.memcache_pool_maxsize = 10 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.876908 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.877083 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.877215 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.memcache_socket_timeout = 1.0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.877377 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.proxies = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.877536 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.tls_allowed_ciphers = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.877695 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.tls_cafile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.877855 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.tls_certfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.878013 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.tls_enabled = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.878176 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] cache.tls_keyfile = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.878349 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep.capabilities = [21, 12, 1, 2, 19] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.878509 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep.group = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.878669 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep.helper_command = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.878831 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep.thread_pool_size = 4 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.878991 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep.user = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.879172 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.879380 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_dhcp_release.group = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.879544 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_dhcp_release.helper_command = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.879703 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_dhcp_release.thread_pool_size = 4 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.879849 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_dhcp_release.user = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.880019 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.880178 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.group = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.880337 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.helper_command = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.880497 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.thread_pool_size = 4 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.880661 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.user = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.880830 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_namespace.capabilities = [21] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.880988 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_namespace.group = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.881146 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_namespace.helper_command = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.881304 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_namespace.thread_pool_size = 4 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.881464 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_namespace.user = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.881632 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_conntrack.capabilities = [12] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.881791 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_conntrack.group = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.881950 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_conntrack.helper_command = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.882109 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_conntrack.thread_pool_size = 4 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.882274 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] privsep_conntrack.user = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.882443 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] AGENT.root_helper = sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.882603 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] AGENT.root_helper_daemon = sudo /usr/local/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.882747 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] AGENT.use_helper_for_ns_read = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.882918 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] SECURITYGROUP.enable_ipset = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.883077 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] SECURITYGROUP.enable_security_group = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.883250 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] SECURITYGROUP.firewall_driver = openvswitch {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.883517 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.883699 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2.extension_drivers = ['port_security'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.883699 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2.external_network_type = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.884018 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2.mechanism_drivers = ['openvswitch', 'hyperv'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.884018 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2.overlay_ip_version = 4 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.884270 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2.path_mtu = 0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.884415 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2.physical_network_mtus = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.884550 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2.tenant_network_types = ['vlan'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.884685 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2.type_drivers = ['local', 'flat', 'vlan', 'gre', 'vxlan', 'geneve'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.884821 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2_type_flat.flat_networks = ['physnet1'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.884944 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2_type_geneve.max_header_size = 30 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.885066 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2_type_geneve.vni_ranges = ['1:1000'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.885222 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2_type_gre.tunnel_id_ranges = ['1:1000'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.885391 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2_type_vlan.network_vlan_ranges = ['physnet1:800:849'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.885558 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2_type_vxlan.vni_ranges = ['1:1000'] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.885717 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ml2_type_vxlan.vxlan_group = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.885886 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] OVS_DRIVER.vnic_type_prohibit_list = [] {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.886056 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.backend = sqlalchemy {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.886215 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.connection = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.886374 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.connection_debug = 0 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.886532 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.connection_parameters = {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.886692 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.connection_recycle_time = 3600 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.886853 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.connection_trace = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.887010 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.db_inc_retry_interval = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.887169 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.db_max_retries = 20 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.887342 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.db_max_retry_interval = 10 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.887500 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.db_retry_interval = 1 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.887661 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.max_overflow = 50 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.887808 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.max_pool_size = 50 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.887968 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.max_retries = 10 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.888127 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.mysql_enable_ndb = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.888286 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.888433 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.pool_timeout = None {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.888591 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.retry_interval = 10 {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.888750 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.slave_connection = **** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.888909 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.sqlite_synchronous = True {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.889067 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] database.use_db_reconnect = False {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 04:14:01.889206 dv-cinder-782405-9-iscsi neutron-server[91536]: DEBUG oslo_service.service [-] ******************************************************************************** {{(pid=91536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 20 04:14:02.046416 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 59192) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:14:02.060541 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [-] 10.222.0.39 "GET / HTTP/1.1" status: 200 len: 253 time: 0.0108068 May 20 04:14:12.692711 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e3df477c-c809-4dc5-80a9-801cd574ec1e None None] get_active_networks_info from dv-cinder-782405-9-iscsi {{(pid=91645) get_active_networks_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:158}} May 20 04:14:12.765868 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-e3df477c-c809-4dc5-80a9-801cd574ec1e None None] No non-hosted networks {{(pid=91645) auto_schedule_networks /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:60}} May 20 04:14:12.780735 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-632a716d-a8b1-4dc0-8bfb-0de2e278d3d0 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988'] for agent, after_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:14.936551 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988'] for agent, after_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:14.938226 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:14.938927 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:14.939482 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:14.940443 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:14.940972 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:14.941499 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:14.942026 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:14.942559 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:14.943088 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:14.943623 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:14.944166 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:14.944694 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:14.945316 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] neutron.api.rpc.handlers.resources_rpc.ResourcesPushToServersRpcApi method report_agent_resource_versions called with arguments (, 'Open vSwitch agent', 'dv-cinder-782405-9-iscsi', {'QosPolicy': '1.8', 'Trunk': '1.1', 'SubPort': '1.0', 'Port': '1.7', 'Subnet': '1.1', 'Network': '1.1', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'Log': '1.0', 'PortForwarding': '1.2', 'ConntrackHelper': '1.0', 'AddressGroup': '1.2'}) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:14:14.951796 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] neutron.api.rpc.handlers.resources_rpc.ResourcesPushToServerRpcCallback method report_agent_resource_versions called with arguments (,) {'agent_type': 'Open vSwitch agent', 'agent_host': 'dv-cinder-782405-9-iscsi', 'version_map': {'QosPolicy': '1.8', 'Trunk': '1.1', 'SubPort': '1.0', 'Port': '1.7', 'Subnet': '1.1', 'Network': '1.1', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'Log': '1.0', 'PortForwarding': '1.2', 'ConntrackHelper': '1.0', 'AddressGroup': '1.2'}} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:14:15.076640 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-fa6f9802-fbad-418b-9151-180573f7697b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988'] for agent, after_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:15.189702 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e6ef956d-b041-463e-92e5-d1c8f24e6360 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:16.153827 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988'] for agent, after_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:17.768453 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988'] for agent, after_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:19.241270 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 59286) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:14:19.844557 dv-cinder-782405-9-iscsi neutron-server[91643]: WARNING oslo_policy.policy [None req-4cb02ff3-8736-48fb-9105-ecfd7ff5cdf0 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 20 04:14:19.845136 dv-cinder-782405-9-iscsi neutron-server[91643]: WARNING oslo_policy.policy [None req-4cb02ff3-8736-48fb-9105-ecfd7ff5cdf0 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 20 04:14:20.053370 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [None req-4cb02ff3-8736-48fb-9105-ecfd7ff5cdf0 admin admin] http://10.222.0.39:9696/v2.0/extensions returned with HTTP 200 {{(pid=91643) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 20 04:14:20.054575 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-4cb02ff3-8736-48fb-9105-ecfd7ff5cdf0 admin admin] 10.222.0.39 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.8113656 May 20 04:14:21.088713 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.db.l3_hamode_db [None req-9950c711-17c2-49d1-8652-ff9f4f13a196 None None] Number of active agents lower than max_l3_agents_per_router. L3 agents available: 1 May 20 04:14:21.172090 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9950c711-17c2-49d1-8652-ff9f4f13a196 None None] Routers set() bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:14:21.172469 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9950c711-17c2-49d1-8652-ff9f4f13a196 None None] Router IDs set() for agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:464}} May 20 04:14:22.378229 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 59300) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:14:22.770313 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-3e3e3d20-560c-4a2c-99a6-4df1197c43c9 admin admin] Request body: {'subnetpool': {'name': 'shared-default-subnetpool-v4', 'default_prefixlen': 26, 'prefixes': ['172.20.0.0/16'], 'shared': True, 'is_default': True}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:14:22.868152 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.policy [None req-3e3e3d20-560c-4a2c-99a6-4df1197c43c9 admin admin] Loaded default policies from ['neutron'] under neutron.policies entry points {{(pid=91643) register_rules /opt/stack/neutron/neutron/policy.py:73}} May 20 04:14:22.912129 dv-cinder-782405-9-iscsi neutron-server[91643]: WARNING oslo_policy.policy [None req-3e3e3d20-560c-4a2c-99a6-4df1197c43c9 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 20 04:14:22.912489 dv-cinder-782405-9-iscsi neutron-server[91643]: WARNING oslo_policy.policy [None req-3e3e3d20-560c-4a2c-99a6-4df1197c43c9 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 20 04:14:22.920940 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.quota [None req-3e3e3d20-560c-4a2c-99a6-4df1197c43c9 admin admin] Loaded quota_driver: . May 20 04:14:22.937257 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-3e3e3d20-560c-4a2c-99a6-4df1197c43c9 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:14:23.005219 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3e3e3d20-560c-4a2c-99a6-4df1197c43c9 admin admin] Notify callbacks ['neutron.objects.securitygroup.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8760692501196', 'neutron.objects.address_group.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8760692446399', 'neutron.objects.network.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8760692454935', 'neutron.objects.qos.policy.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8760692452731', 'neutron.objects.subnetpool.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8760692470982', 'neutron.objects.address_scope.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8760692469348', 'neutron.plugins.ml2.plugin.External_net_db_mixin._process_ext_policy_create-593644', 'neutron.plugins.ml2.plugin.NeutronDbPluginV2.validate_network_rbac_policy_change-7835821', 'neutron.services.network_ip_availability.plugin.NeutronDbPluginV2.validate_network_rbac_policy_change-286902', 'neutron.services.network_ip_availability.plugin.NeutronDbPluginV2.validate_network_rbac_policy_change-429562'] for rbac-policy, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:23.074458 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3e3e3d20-560c-4a2c-99a6-4df1197c43c9 admin admin] Notify callbacks [] for subnetpool, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:23.075678 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-3e3e3d20-560c-4a2c-99a6-4df1197c43c9 admin admin] 10.222.0.39 "POST /v2.0/subnetpools HTTP/1.1" status: 201 len: 739 time: 0.6960859 May 20 04:14:25.681913 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 59314) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:14:26.033377 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Request body: {'network': {'tenant_id': '61e4cb63ed58450b84b65106d258ea76', 'admin_state_up': True, 'name': 'private', 'project_id': '61e4cb63ed58450b84b65106d258ea76'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:14:26.038219 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:14:26.042738 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Usage tracker for resource:network and tenant:61e4cb63ed58450b84b65106d258ea76 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:14:26.047573 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:14:26.053991 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:14:26.066091 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Made reservation on behalf of 61e4cb63ed58450b84b65106d258ea76 for: {'network': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:14:26.069760 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-585852'] for network, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:26.088451 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:26.102864 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:14:26.106930 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Usage tracker for resource:security_group_rule and tenant:61e4cb63ed58450b84b65106d258ea76 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:14:26.112512 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:14:26.121614 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:14:26.225581 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:26.232740 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:26.239468 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:14:26.262171 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 819} {{(pid=91643) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 20 04:14:26.288431 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Notify callbacks ['neutron.services.segments.db._add_segment_host_mapping_for_segment-8760688960970', 'neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:26.289028 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Unsubscribe: segment precommit_create {{(pid=91643) unsubscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:83}} May 20 04:14:26.289486 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.segments_db [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Added segment 6879b57a-f9a6-4d4e-b5a0-dc24c6cb79e2 of type vlan for network 65c1116d-8a94-470d-8f12-ff3ae63f0c9d May 20 04:14:26.335953 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=bc9dd51f-eb17-416a-abbe-aebcf09a4b28,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:14:26.356937 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:26.357286 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:26.357613 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:26.357924 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:26.358235 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:26.358560 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:26.358873 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:26.359178 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:26.359530 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:26.359839 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:26.360125 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:26.360427 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Tracked resource versions report: May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: pushed versions: May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: { 'AddressGroup': {'1.2'}, May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'ConntrackHelper': {'1.0'}, May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'Log': {'1.0'}, May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'Network': {'1.1'}, May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'Port': {'1.7'}, May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'PortForwarding': {'1.2'}, May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'QosPolicy': {'1.8'}, May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'SecurityGroup': {'1.4'}, May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'SecurityGroupRule': {'1.2'}, May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'SubPort': {'1.0'}, May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'Subnet': {'1.1'}, May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'Trunk': {'1.1'}} May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: consumer versions: May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: { DHCP agent@dv-cinder-782405-9-iscsi: {}, May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: L3 agent@dv-cinder-782405-9-iscsi: {}, May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: Metadata agent@dv-cinder-782405-9-iscsi: {}, May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: Metering agent@dv-cinder-782405-9-iscsi: {}, May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: Open vSwitch agent@dv-cinder-782405-9-iscsi: { 'AddressGroup': '1.2', May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'ConntrackHelper': '1.0', May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'Log': '1.0', May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'Network': '1.1', May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'Port': '1.7', May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'PortForwarding': '1.2', May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'QosPolicy': '1.8', May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'SecurityGroup': '1.4', May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'SecurityGroupRule': '1.2', May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'SubPort': '1.0', May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'Subnet': '1.1', May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: 'Trunk': '1.1'}} May 20 04:14:26.361536 dv-cinder-782405-9-iscsi neutron-server[91643]: {{(pid=91643) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 20 04:14:26.405659 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.166s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:14:26.407347 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8760688516720'] for network, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:26.449604 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915'] for network, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:26.451268 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:14:26.451883 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:26.456820 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:14:26.555197 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] 10.222.0.39 "POST /v2.0/networks HTTP/1.1" status: 201 len: 849 time: 0.8720775 May 20 04:14:26.560394 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Pushing event updated for resources: {'Network': ['ID=65c1116d-8a94-470d-8f12-ff3ae63f0c9d,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:14:26.564112 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.107s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:14:26.576060 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Network 65c1116d-8a94-470d-8f12-ff3ae63f0c9d details requested by agent ovs-agent-dv-cinder-782405-9-iscsi with host dv-cinder-782405-9-iscsi {{(pid=91645) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 20 04:14:29.209335 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 59330) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:14:29.857773 dv-cinder-782405-9-iscsi neutron-server[91644]: WARNING oslo_policy.policy [None req-2602d62c-1d8b-4d09-abbd-f68f3354208c admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 20 04:14:29.858370 dv-cinder-782405-9-iscsi neutron-server[91644]: WARNING oslo_policy.policy [None req-2602d62c-1d8b-4d09-abbd-f68f3354208c admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 20 04:14:30.290612 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.policy [None req-2602d62c-1d8b-4d09-abbd-f68f3354208c admin admin] Loaded default policies from ['neutron'] under neutron.policies entry points {{(pid=91644) register_rules /opt/stack/neutron/neutron/policy.py:73}} May 20 04:14:30.335858 dv-cinder-782405-9-iscsi neutron-server[91644]: WARNING oslo_policy.policy [None req-2602d62c-1d8b-4d09-abbd-f68f3354208c admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 20 04:14:30.336164 dv-cinder-782405-9-iscsi neutron-server[91644]: WARNING oslo_policy.policy [None req-2602d62c-1d8b-4d09-abbd-f68f3354208c admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 20 04:14:30.345204 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-2602d62c-1d8b-4d09-abbd-f68f3354208c admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:14:30.346969 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-2602d62c-1d8b-4d09-abbd-f68f3354208c admin admin] 10.222.0.39 "GET /v2.0/networks/65c1116d-8a94-470d-8f12-ff3ae63f0c9d HTTP/1.1" status: 200 len: 825 time: 1.1364110 May 20 04:14:30.381842 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-ddcfb723-1ef0-4567-8372-f96a8df1bb13 admin admin] 10.222.0.39 "GET /v2.0/subnetpools/b8e14113-efb9-487e-a28c-298285e213da HTTP/1.1" status: 200 len: 734 time: 0.0266705 May 20 04:14:30.390346 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Request body: {'subnet': {'ip_version': 4, 'name': 'private-subnet', 'gateway_ip': '172.20.0.1', 'network_id': '65c1116d-8a94-470d-8f12-ff3ae63f0c9d', 'subnetpool_id': 'b8e14113-efb9-487e-a28c-298285e213da', 'tenant_id': '61e4cb63ed58450b84b65106d258ea76'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:14:30.392001 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.quota [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Loaded quota_driver: . May 20 04:14:30.398438 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:14:30.404271 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Usage tracker for resource:subnet and tenant:61e4cb63ed58450b84b65106d258ea76 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:14:30.407802 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:14:30.415428 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:14:30.433218 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Made reservation on behalf of 61e4cb63ed58450b84b65106d258ea76 for: {'subnet': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:14:30.436263 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Notify callbacks [] for subnet, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:30.963516 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868'] for subnet, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:30.968104 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:14:31.049058 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Checking segment: {'id': '6879b57a-f9a6-4d4e-b5a0-dc24c6cb79e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 819, 'network_id': '65c1116d-8a94-470d-8f12-ff3ae63f0c9d'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:14:31.049457 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Network hostable DHCP agents. Network: 65c1116d-8a94-470d-8f12-ff3ae63f0c9d, hostable agents: ['fffae25a-f450-4182-8f9e-ac29d143f6c9'], hosted agents: [] {{(pid=91644) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 20 04:14:31.059190 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Pushing event updated for resources: {'Subnet': ['ID=10045323-b78d-4e34-b0af-361098232ecf,revision_number=0']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:14:31.068569 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:31.068917 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:31.069240 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:31.069550 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:31.069862 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:31.070184 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:31.070489 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:31.070791 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:31.071085 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:31.071466 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:31.071772 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:31.072054 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Tracked resource versions report: May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: pushed versions: May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: { 'AddressGroup': {'1.2'}, May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'ConntrackHelper': {'1.0'}, May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'Log': {'1.0'}, May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'Network': {'1.1'}, May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'Port': {'1.7'}, May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'PortForwarding': {'1.2'}, May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'QosPolicy': {'1.8'}, May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'SecurityGroup': {'1.4'}, May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'SecurityGroupRule': {'1.2'}, May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'SubPort': {'1.0'}, May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'Subnet': {'1.1'}, May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'Trunk': {'1.1'}} May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: consumer versions: May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: { DHCP agent@dv-cinder-782405-9-iscsi: {}, May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: L3 agent@dv-cinder-782405-9-iscsi: {}, May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: Metadata agent@dv-cinder-782405-9-iscsi: {}, May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: Metering agent@dv-cinder-782405-9-iscsi: {}, May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: Open vSwitch agent@dv-cinder-782405-9-iscsi: { 'AddressGroup': '1.2', May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'ConntrackHelper': '1.0', May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'Log': '1.0', May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'Network': '1.1', May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'Port': '1.7', May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'PortForwarding': '1.2', May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'QosPolicy': '1.8', May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'SecurityGroup': '1.4', May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'SecurityGroupRule': '1.2', May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'SubPort': '1.0', May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'Subnet': '1.1', May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: 'Trunk': '1.1'}} May 20 04:14:31.073136 dv-cinder-782405-9-iscsi neutron-server[91644]: {{(pid=91644) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 20 04:14:31.104417 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Network 65c1116d-8a94-470d-8f12-ff3ae63f0c9d is scheduled to be hosted by DHCP agent fffae25a-f450-4182-8f9e-ac29d143f6c9 with binding_index 1 {{(pid=91644) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 20 04:14:31.110858 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.143s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:14:31.116849 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-782405-9-iscsi, 65c1116d-8a94-470d-8f12-ff3ae63f0c9d) {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:14:31.122199 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:14:31.123413 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:31.129218 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-20fa81d1-04c8-47fa-929a-8989e11f901d None None] Network 65c1116d-8a94-470d-8f12-ff3ae63f0c9d requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:14:31.146513 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] 10.222.0.39 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 854 time: 0.7594769 May 20 04:14:31.443512 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcp85c0b5c8-c799-5455-967d-3e9c2c8b4839-65c1116d-8a94-470d-8f12-ff3ae63f0c9d', 'network_id': '65c1116d-8a94-470d-8f12-ff3ae63f0c9d', 'tenant_id': '61e4cb63ed58450b84b65106d258ea76', 'fixed_ips': [{'subnet_id': '10045323-b78d-4e34-b0af-361098232ecf'}]}} from dv-cinder-782405-9-iscsi. {{(pid=91645) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 20 04:14:31.444670 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:31.629974 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.db_base_plugin_common [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Allocated IP 172.20.0.2 (65c1116d-8a94-470d-8f12-ff3ae63f0c9d/10045323-b78d-4e34-b0af-361098232ecf/03798bed-9045-4ff6-8d71-a4df490fc833) {{(pid=91645) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:14:31.860775 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Notify callbacks [] for port, precommit_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:31.874758 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Transition to ACTIVE for port object 03798bed-9045-4ff6-8d71-a4df490fc833 will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:14:31.894896 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Transition to ACTIVE for port object 03798bed-9045-4ff6-8d71-a4df490fc833 will not be triggered until provisioned by entity DHCP. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:14:31.902142 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:31.907662 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:14:31.967605 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Network 65c1116d-8a94-470d-8f12-ff3ae63f0c9d is already hosted by enough agents. {{(pid=91645) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:14:31.970845 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.base_scheduler [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Resources bound (agent type, host, resource id): {{(pid=91645) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:14:31.977861 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Received port after_create {{(pid=91645) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:14:31.983785 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Pushing event updated for resources: {'Port': ['ID=03798bed-9045-4ff6-8d71-a4df490fc833,revision_number=1']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:14:32.023380 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:32.023701 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:32.024025 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:32.024343 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:32.024652 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:32.024969 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:32.025267 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:32.025574 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:32.025904 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:32.026192 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:32.026494 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:32.026796 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Tracked resource versions report: May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: pushed versions: May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: { 'AddressGroup': {'1.2'}, May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'ConntrackHelper': {'1.0'}, May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'Log': {'1.0'}, May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'Network': {'1.1'}, May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'Port': {'1.7'}, May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'PortForwarding': {'1.2'}, May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'QosPolicy': {'1.8'}, May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'SecurityGroup': {'1.4'}, May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'SecurityGroupRule': {'1.2'}, May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'SubPort': {'1.0'}, May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'Subnet': {'1.1'}, May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'Trunk': {'1.1'}} May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: consumer versions: May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: { DHCP agent@dv-cinder-782405-9-iscsi: {}, May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: L3 agent@dv-cinder-782405-9-iscsi: {}, May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: Metadata agent@dv-cinder-782405-9-iscsi: {}, May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: Metering agent@dv-cinder-782405-9-iscsi: {}, May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: Open vSwitch agent@dv-cinder-782405-9-iscsi: { 'AddressGroup': '1.2', May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'ConntrackHelper': '1.0', May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'Log': '1.0', May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'Network': '1.1', May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'Port': '1.7', May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'PortForwarding': '1.2', May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'QosPolicy': '1.8', May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'SecurityGroup': '1.4', May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'SecurityGroupRule': '1.2', May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'SubPort': '1.0', May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'Subnet': '1.1', May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: 'Trunk': '1.1'}} May 20 04:14:32.027944 dv-cinder-782405-9-iscsi neutron-server[91645]: {{(pid=91645) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 20 04:14:32.036064 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.129s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:14:32.085134 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Attempting to bind port 03798bed-9045-4ff6-8d71-a4df490fc833 on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:14:32.085526 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Attempting to bind port 03798bed-9045-4ff6-8d71-a4df490fc833 by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '6879b57a-f9a6-4d4e-b5a0-dc24c6cb79e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 819, 'network_id': '65c1116d-8a94-470d-8f12-ff3ae63f0c9d'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:14:32.085947 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Attempting to bind port 03798bed-9045-4ff6-8d71-a4df490fc833 on network 65c1116d-8a94-470d-8f12-ff3ae63f0c9d {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:14:32.091781 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 14, 15), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:14:32.092183 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Checking segment: {'id': '6879b57a-f9a6-4d4e-b5a0-dc24c6cb79e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 819, 'network_id': '65c1116d-8a94-470d-8f12-ff3ae63f0c9d'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:14:32.092651 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:32.093043 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Bound using segment: {'id': '6879b57a-f9a6-4d4e-b5a0-dc24c6cb79e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 819, 'network_id': '65c1116d-8a94-470d-8f12-ff3ae63f0c9d'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:14:32.093533 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Bound port: 03798bed-9045-4ff6-8d71-a4df490fc833, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '6879b57a-f9a6-4d4e-b5a0-dc24c6cb79e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 819, 'network_id': '65c1116d-8a94-470d-8f12-ff3ae63f0c9d'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:14:32.093902 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:32.183881 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] For port 03798bed-9045-4ff6-8d71-a4df490fc833, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:14:32.188085 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] For port 03798bed-9045-4ff6-8d71-a4df490fc833, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=03798bed-9045-4ff6-8d71-a4df490fc833,segment=NetworkSegment(6879b57a-f9a6-4d4e-b5a0-dc24c6cb79e2),segment_id=6879b57a-f9a6-4d4e-b5a0-dc24c6cb79e2)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:14:32.192922 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:32.196802 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:14:32.286051 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Pushing event updated for resources: {'Port': ['ID=03798bed-9045-4ff6-8d71-a4df490fc833,revision_number=2']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:14:32.296189 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.099s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:14:33.479646 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 59350) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:14:33.859692 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [None req-6a3bdbf1-cb15-4e50-992f-47813ac2962f admin admin] http://10.222.0.39:9696/v2.0/extensions returned with HTTP 200 {{(pid=91644) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 20 04:14:33.860702 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-6a3bdbf1-cb15-4e50-992f-47813ac2962f admin admin] 10.222.0.39 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.3802340 May 20 04:14:36.163803 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 59360) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:14:36.558043 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [None req-4cdc2f18-82aa-45db-9725-93bf39fe65f8 admin admin] http://10.222.0.39:9696/v2.0/extensions returned with HTTP 200 {{(pid=91644) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 20 04:14:36.559008 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-4cdc2f18-82aa-45db-9725-93bf39fe65f8 admin admin] 10.222.0.39 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.3942993 May 20 04:14:36.839734 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 03798bed-9045-4ff6-8d71-a4df490fc833 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:14:36.914068 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port 03798bed-9045-4ff6-8d71-a4df490fc833 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:14:37.635965 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e9540f5a-6465-4f7b-968d-1b8cf477c53a None None] Network 65c1116d-8a94-470d-8f12-ff3ae63f0c9d requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:14:37.746310 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-dddd9bb9-97b5-40b8-8347-e455b4eaaea2 None None] Provisioning for port 03798bed-9045-4ff6-8d71-a4df490fc833 completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:14:37.755806 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-dddd9bb9-97b5-40b8-8347-e455b4eaaea2 None None] Provisioning complete for port 03798bed-9045-4ff6-8d71-a4df490fc833 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:14:37.756204 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-dddd9bb9-97b5-40b8-8347-e455b4eaaea2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:37.967336 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-dddd9bb9-97b5-40b8-8347-e455b4eaaea2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:38.006568 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-dddd9bb9-97b5-40b8-8347-e455b4eaaea2 None None] For port 03798bed-9045-4ff6-8d71-a4df490fc833, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=03798bed-9045-4ff6-8d71-a4df490fc833,segment=NetworkSegment(6879b57a-f9a6-4d4e-b5a0-dc24c6cb79e2),segment_id=6879b57a-f9a6-4d4e-b5a0-dc24c6cb79e2)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:14:38.014604 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-dddd9bb9-97b5-40b8-8347-e455b4eaaea2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:38.017112 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:14:38.088797 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dddd9bb9-97b5-40b8-8347-e455b4eaaea2 None None] Pushing event updated for resources: {'Port': ['ID=03798bed-9045-4ff6-8d71-a4df490fc833,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:14:38.091965 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-dddd9bb9-97b5-40b8-8347-e455b4eaaea2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.075s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:14:38.242386 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-bc3c7bc7-60b1-44f7-bf0b-4e770b7c839d None None] Provisioning complete for port 03798bed-9045-4ff6-8d71-a4df490fc833 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:14:38.243019 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-bc3c7bc7-60b1-44f7-bf0b-4e770b7c839d None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:38.782018 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 5 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:14:38.816173 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 03798bed-9045-4ff6-8d71-a4df490fc833 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:14:38.904308 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 03798bed-9045-4ff6-8d71-a4df490fc833 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:14:38.904647 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:39.188423 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 59376) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:14:39.565373 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-8bf4b324-70ca-4ad6-9a76-de2cd5259f58 admin admin] Request body: {'router': {'tenant_id': '61e4cb63ed58450b84b65106d258ea76', 'admin_state_up': True, 'name': 'router1'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:14:39.573807 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-8bf4b324-70ca-4ad6-9a76-de2cd5259f58 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:14:39.580318 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-8bf4b324-70ca-4ad6-9a76-de2cd5259f58 admin admin] Usage tracker for resource:router and tenant:61e4cb63ed58450b84b65106d258ea76 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:14:39.583804 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-8bf4b324-70ca-4ad6-9a76-de2cd5259f58 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:14:39.592562 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-8bf4b324-70ca-4ad6-9a76-de2cd5259f58 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:14:39.607416 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8bf4b324-70ca-4ad6-9a76-de2cd5259f58 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-8071540', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-260712'] for router, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:39.609046 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8bf4b324-70ca-4ad6-9a76-de2cd5259f58 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-8056436', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-8053301', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-7881139', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-260707'] for router, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:39.630719 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8bf4b324-70ca-4ad6-9a76-de2cd5259f58 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:39.742996 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8bf4b324-70ca-4ad6-9a76-de2cd5259f58 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-8071551', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-8053391'] for router, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:39.764746 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8bf4b324-70ca-4ad6-9a76-de2cd5259f58 admin admin] Notify callbacks [] for router, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:39.766391 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-8bf4b324-70ca-4ad6-9a76-de2cd5259f58 admin admin] 10.222.0.39 "POST /v2.0/routers HTTP/1.1" status: 201 len: 709 time: 0.5768352 May 20 04:14:42.118644 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 59386) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:14:42.510970 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [None req-ff43a9ed-770a-4438-8acc-163493409407 admin admin] http://10.222.0.39:9696/v2.0/extensions returned with HTTP 200 {{(pid=91643) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 20 04:14:42.511961 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-ff43a9ed-770a-4438-8acc-163493409407 admin admin] 10.222.0.39 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.3924479 May 20 04:14:42.591779 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-a1a59063-eab9-4667-a064-fea99bd93e57 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:44.833543 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 59396) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:14:44.926568 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:45.061797 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-b39a574f-525d-4855-86ce-40938134ff44 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:45.228421 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Request body: {'network': {'admin_state_up': True, 'name': 'public', 'is_default': True, 'router:external': True, 'provider:network_type': 'flat', 'provider:physical_network': 'physnet1'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:14:45.235828 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:14:45.242575 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Usage tracker for resource:network and tenant:2361ee7db41742a7b41efa29b36d3870 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:14:45.245719 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:14:45.252016 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:14:45.262642 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Made reservation on behalf of 2361ee7db41742a7b41efa29b36d3870 for: {'network': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:14:45.266198 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-585852'] for network, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:45.298983 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:45.313859 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:14:45.317875 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Usage tracker for resource:security_group_rule and tenant:2361ee7db41742a7b41efa29b36d3870 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:14:45.321252 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:14:45.327325 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:14:45.423688 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:45.428939 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:45.432056 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:14:45.452711 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.type_flat [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Reserving flat network on physical network physnet1 {{(pid=91644) reserve_provider_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_flat.py:100}} May 20 04:14:45.465512 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=ba24e97d-aa73-4212-9a79-fc39296f8035,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:14:45.472636 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.040s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:14:45.473652 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Notify callbacks ['neutron.services.segments.db._add_segment_host_mapping_for_segment-8760688960970', 'neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:45.474193 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Unsubscribe: segment precommit_create {{(pid=91644) unsubscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:83}} May 20 04:14:45.474607 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.db.segments_db [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Added segment 240fc062-60cd-4ee1-8e1d-b9956ca52b7b of type flat for network 712ce8c4-8feb-4719-bf13-e06992125cd9 May 20 04:14:45.528157 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8760688516720'] for network, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:45.574041 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915'] for network, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:45.575413 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:14:45.576036 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:45.579396 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:14:45.674272 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Pushing event updated for resources: {'Network': ['ID=712ce8c4-8feb-4719-bf13-e06992125cd9,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:14:45.676965 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] 10.222.0.39 "POST /v2.0/networks HTTP/1.1" status: 201 len: 847 time: 0.8421121 May 20 04:14:45.692846 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.113s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:14:45.700451 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 details requested by agent ovs-agent-dv-cinder-782405-9-iscsi with host dv-cinder-782405-9-iscsi {{(pid=91645) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 20 04:14:46.106808 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:47.727915 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:47.994819 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 59416) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:14:48.235551 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-0586a4ed-713f-41c1-b6bd-2bfe01a93a58 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:14:48.236778 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-0586a4ed-713f-41c1-b6bd-2bfe01a93a58 admin admin] 10.222.0.39 "GET /v2.0/subnets/10045323-b78d-4e34-b0af-361098232ecf HTTP/1.1" status: 200 len: 849 time: 0.2410126 May 20 04:14:48.424331 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-a1ef0857-acbd-4e66-a829-c3a97435fd2b admin admin] 10.222.0.39 "GET /v2.0/routers/ae96cfaf-58ff-48ee-bb34-133e5546ed99 HTTP/1.1" status: 200 len: 704 time: 0.1791475 May 20 04:14:48.911308 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:49.161888 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Allocated IP 172.20.0.1 (65c1116d-8a94-470d-8f12-ff3ae63f0c9d/10045323-b78d-4e34-b0af-361098232ecf/ab0a3324-2266-4f95-bb12-ee62475ae67c) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:14:49.324481 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:49.343403 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.provisioning_blocks [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Transition to ACTIVE for port object ab0a3324-2266-4f95-bb12-ee62475ae67c will not be triggered until provisioned by entity DHCP. {{(pid=91643) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:14:49.402692 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:49.409099 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:14:49.469348 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Network 65c1116d-8a94-470d-8f12-ff3ae63f0c9d is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:14:49.472634 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:14:49.478308 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:14:49.478809 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-8003132'] for router_interface, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:49.601627 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Pushing event updated for resources: {'Port': ['ID=ab0a3324-2266-4f95-bb12-ee62475ae67c,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:14:49.610680 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.201s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:14:49.665920 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-0ddd6842-de35-4190-9fba-59fc7c459f77 None None] Provisioning for port ab0a3324-2266-4f95-bb12-ee62475ae67c completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:14:49.671177 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-0ddd6842-de35-4190-9fba-59fc7c459f77 None None] Provisioning complete for port ab0a3324-2266-4f95-bb12-ee62475ae67c triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:14:49.671617 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0ddd6842-de35-4190-9fba-59fc7c459f77 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:49.724532 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-0ddd6842-de35-4190-9fba-59fc7c459f77 None None] Port ab0a3324-2266-4f95-bb12-ee62475ae67c cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:14:49.736639 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:49.997326 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Notify callbacks [] for port, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:50.004495 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:50.005740 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-7924897', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-8003163'] for router_interface, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:50.008782 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:14:50.137416 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Pushing event updated for resources: {'Port': ['ID=ab0a3324-2266-4f95-bb12-ee62475ae67c,revision_number=2']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:14:50.143889 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.135s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:14:50.198320 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Network 65c1116d-8a94-470d-8f12-ff3ae63f0c9d is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:14:50.200223 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:14:50.347902 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-bddf6f39-487f-4973-8b48-937a1e332e06 None None] Provisioning complete for port ab0a3324-2266-4f95-bb12-ee62475ae67c triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:14:50.348266 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-bddf6f39-487f-4973-8b48-937a1e332e06 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:50.390295 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Router ae96cfaf-58ff-48ee-bb34-133e5546ed99 is scheduled to L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 with binding_index 1 {{(pid=91643) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 20 04:14:50.402761 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-bddf6f39-487f-4973-8b48-937a1e332e06 None None] Port ab0a3324-2266-4f95-bb12-ee62475ae67c cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:14:50.544840 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:14:50.554385 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] 10.222.0.39 "PUT /v2.0/routers/ae96cfaf-58ff-48ee-bb34-133e5546ed99/add_router_interface HTTP/1.1" status: 200 len: 508 time: 2.1225433 May 20 04:14:50.555368 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Sync routers for ids ['ae96cfaf-58ff-48ee-bb34-133e5546ed99'] in dv-cinder-782405-9-iscsi {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:14:50.566524 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Routers {'ae96cfaf-58ff-48ee-bb34-133e5546ed99'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:14:50.567389 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 0, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:14:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['ae96cfaf-58ff-48ee-bb34-133e5546ed99'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:14:50.739973 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Set router ae96cfaf-58ff-48ee-bb34-133e5546ed99 gateway port host: None {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:14:51.100801 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'ae96cfaf-58ff-48ee-bb34-133e5546ed99', 'name': 'router1', 'tenant_id': '61e4cb63ed58450b84b65106d258ea76', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:14:39Z', 'updated_at': '2021-05-20T04:14:49Z', 'revision_number': 2, 'project_id': '61e4cb63ed58450b84b65106d258ea76', 'gw_port_host': None, '_interfaces': [{'id': 'ab0a3324-2266-4f95-bb12-ee62475ae67c', 'name': '', 'network_id': '65c1116d-8a94-470d-8f12-ff3ae63f0c9d', 'tenant_id': '61e4cb63ed58450b84b65106d258ea76', 'mac_address': 'fa:16:3e:b3:e0:82', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'ae96cfaf-58ff-48ee-bb34-133e5546ed99', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '10045323-b78d-4e34-b0af-361098232ecf', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:14:48Z', 'updated_at': '2021-05-20T04:14:49Z', 'revision_number': 2, 'project_id': '61e4cb63ed58450b84b65106d258ea76', 'subnets': [{'id': '10045323-b78d-4e34-b0af-361098232ecf', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': 'b8e14113-efb9-487e-a28c-298285e213da'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:14:51.105969 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Checking router: ae96cfaf-58ff-48ee-bb34-133e5546ed99 for host: dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:14:51.106354 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Updating router ae96cfaf-58ff-48ee-bb34-133e5546ed99 port ab0a3324-2266-4f95-bb12-ee62475ae67c binding host dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 20 04:14:51.158641 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:51.538645 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Notify callbacks [] for port, precommit_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:51.553273 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.provisioning_blocks [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Transition to ACTIVE for port object ab0a3324-2266-4f95-bb12-ee62475ae67c will not be triggered until provisioned by entity L2. {{(pid=91646) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:14:51.560106 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:51.565881 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:14:51.791449 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6d0351fa-a266-4e28-b4ff-583cbf6bb130 None None] Pushing event updated for resources: {'Port': ['ID=ab0a3324-2266-4f95-bb12-ee62475ae67c,revision_number=3']} {{(pid=91646) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:14:51.803523 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6d0351fa-a266-4e28-b4ff-583cbf6bb130 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:51.803849 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6d0351fa-a266-4e28-b4ff-583cbf6bb130 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:51.804166 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6d0351fa-a266-4e28-b4ff-583cbf6bb130 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:51.804474 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6d0351fa-a266-4e28-b4ff-583cbf6bb130 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:51.804775 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6d0351fa-a266-4e28-b4ff-583cbf6bb130 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:51.805084 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6d0351fa-a266-4e28-b4ff-583cbf6bb130 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:51.805383 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6d0351fa-a266-4e28-b4ff-583cbf6bb130 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:51.805680 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6d0351fa-a266-4e28-b4ff-583cbf6bb130 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:51.805984 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6d0351fa-a266-4e28-b4ff-583cbf6bb130 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:51.806289 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6d0351fa-a266-4e28-b4ff-583cbf6bb130 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:51.806587 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6d0351fa-a266-4e28-b4ff-583cbf6bb130 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:51.806864 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6d0351fa-a266-4e28-b4ff-583cbf6bb130 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-6d0351fa-a266-4e28-b4ff-583cbf6bb130 None None] Tracked resource versions report: May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: pushed versions: May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: { 'AddressGroup': {'1.2'}, May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'ConntrackHelper': {'1.0'}, May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'Log': {'1.0'}, May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'Network': {'1.1'}, May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'Port': {'1.7'}, May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'PortForwarding': {'1.2'}, May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'QosPolicy': {'1.8'}, May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'SecurityGroup': {'1.4'}, May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'SecurityGroupRule': {'1.2'}, May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'SubPort': {'1.0'}, May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'Subnet': {'1.1'}, May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'Trunk': {'1.1'}} May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: consumer versions: May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: { DHCP agent@dv-cinder-782405-9-iscsi: {}, May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: L3 agent@dv-cinder-782405-9-iscsi: {}, May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: Metadata agent@dv-cinder-782405-9-iscsi: {}, May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: Metering agent@dv-cinder-782405-9-iscsi: {}, May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: Open vSwitch agent@dv-cinder-782405-9-iscsi: { 'AddressGroup': '1.2', May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'ConntrackHelper': '1.0', May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'Log': '1.0', May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'Network': '1.1', May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'Port': '1.7', May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'PortForwarding': '1.2', May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'QosPolicy': '1.8', May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'SecurityGroup': '1.4', May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'SecurityGroupRule': '1.2', May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'SubPort': '1.0', May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'Subnet': '1.1', May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: 'Trunk': '1.1'}} May 20 04:14:51.807971 dv-cinder-782405-9-iscsi neutron-server[91646]: {{(pid=91646) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 20 04:14:51.811997 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Attempting to bind port ab0a3324-2266-4f95-bb12-ee62475ae67c on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:14:51.812413 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Attempting to bind port ab0a3324-2266-4f95-bb12-ee62475ae67c by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '6879b57a-f9a6-4d4e-b5a0-dc24c6cb79e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 819, 'network_id': '65c1116d-8a94-470d-8f12-ff3ae63f0c9d'}] {{(pid=91646) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:14:51.812908 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Attempting to bind port ab0a3324-2266-4f95-bb12-ee62475ae67c on network 65c1116d-8a94-470d-8f12-ff3ae63f0c9d {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:14:51.815072 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-6d0351fa-a266-4e28-b4ff-583cbf6bb130 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.249s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:14:51.819105 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 14, 45), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:14:51.819335 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Checking segment: {'id': '6879b57a-f9a6-4d4e-b5a0-dc24c6cb79e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 819, 'network_id': '65c1116d-8a94-470d-8f12-ff3ae63f0c9d'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91646) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:14:51.819783 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:51.820149 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Bound using segment: {'id': '6879b57a-f9a6-4d4e-b5a0-dc24c6cb79e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 819, 'network_id': '65c1116d-8a94-470d-8f12-ff3ae63f0c9d'} {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:14:51.820629 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Bound port: ab0a3324-2266-4f95-bb12-ee62475ae67c, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '6879b57a-f9a6-4d4e-b5a0-dc24c6cb79e2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 819, 'network_id': '65c1116d-8a94-470d-8f12-ff3ae63f0c9d'}}] {{(pid=91646) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:14:51.820986 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:51.908131 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.db [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] For port ab0a3324-2266-4f95-bb12-ee62475ae67c, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91646) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:14:51.912380 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.db [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] For port ab0a3324-2266-4f95-bb12-ee62475ae67c, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=ab0a3324-2266-4f95-bb12-ee62475ae67c,segment=NetworkSegment(6879b57a-f9a6-4d4e-b5a0-dc24c6cb79e2),segment_id=6879b57a-f9a6-4d4e-b5a0-dc24c6cb79e2)] {{(pid=91646) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:14:51.917586 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:51.922180 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-6d0351fa-a266-4e28-b4ff-583cbf6bb130 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:14:52.004298 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Routers {'ae96cfaf-58ff-48ee-bb34-133e5546ed99'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:14:52.004924 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 0, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:14:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['ae96cfaf-58ff-48ee-bb34-133e5546ed99'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:14:52.057920 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6d0351fa-a266-4e28-b4ff-583cbf6bb130 None None] Pushing event updated for resources: {'Port': ['ID=ab0a3324-2266-4f95-bb12-ee62475ae67c,revision_number=4']} {{(pid=91646) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:14:52.070251 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-6d0351fa-a266-4e28-b4ff-583cbf6bb130 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.148s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:14:52.158797 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] Set router ae96cfaf-58ff-48ee-bb34-133e5546ed99 gateway port host: None {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:14:52.477071 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'ae96cfaf-58ff-48ee-bb34-133e5546ed99', 'name': 'router1', 'tenant_id': '61e4cb63ed58450b84b65106d258ea76', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:14:39Z', 'updated_at': '2021-05-20T04:14:49Z', 'revision_number': 2, 'project_id': '61e4cb63ed58450b84b65106d258ea76', 'gw_port_host': None, '_interfaces': [{'id': 'ab0a3324-2266-4f95-bb12-ee62475ae67c', 'name': '', 'network_id': '65c1116d-8a94-470d-8f12-ff3ae63f0c9d', 'tenant_id': '61e4cb63ed58450b84b65106d258ea76', 'mac_address': 'fa:16:3e:b3:e0:82', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'ae96cfaf-58ff-48ee-bb34-133e5546ed99', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '10045323-b78d-4e34-b0af-361098232ecf', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:14:48Z', 'updated_at': '2021-05-20T04:14:51Z', 'revision_number': 4, 'project_id': '61e4cb63ed58450b84b65106d258ea76', 'subnets': [{'id': '10045323-b78d-4e34-b0af-361098232ecf', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': 'b8e14113-efb9-487e-a28c-298285e213da'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:14:52.480960 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0fd6f03b-4109-4fbc-82c6-97985c222f6c None None] The sync data for ids ['ae96cfaf-58ff-48ee-bb34-133e5546ed99'] in dv-cinder-782405-9-iscsi is: [{'id': 'ae96cfaf-58ff-48ee-bb34-133e5546ed99', 'name': 'router1', 'tenant_id': '61e4cb63ed58450b84b65106d258ea76', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:14:39Z', 'updated_at': '2021-05-20T04:14:49Z', 'revision_number': 2, 'project_id': '61e4cb63ed58450b84b65106d258ea76', 'gw_port_host': None, '_interfaces': [{'id': 'ab0a3324-2266-4f95-bb12-ee62475ae67c', 'name': '', 'network_id': '65c1116d-8a94-470d-8f12-ff3ae63f0c9d', 'tenant_id': '61e4cb63ed58450b84b65106d258ea76', 'mac_address': 'fa:16:3e:b3:e0:82', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'ae96cfaf-58ff-48ee-bb34-133e5546ed99', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '10045323-b78d-4e34-b0af-361098232ecf', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:14:48Z', 'updated_at': '2021-05-20T04:14:51Z', 'revision_number': 4, 'project_id': '61e4cb63ed58450b84b65106d258ea76', 'subnets': [{'id': '10045323-b78d-4e34-b0af-361098232ecf', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': 'b8e14113-efb9-487e-a28c-298285e213da'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:14:52.676705 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 59438) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:14:52.823533 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-de4d0f10-5191-4126-8cc0-20180f8869b4 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:14:52.824902 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-de4d0f10-5191-4126-8cc0-20180f8869b4 admin admin] 10.222.0.39 "GET /v2.0/networks/712ce8c4-8feb-4719-bf13-e06992125cd9 HTTP/1.1" status: 200 len: 842 time: 0.1472588 May 20 04:14:52.834335 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-9c6d9180-1c3b-47b3-99b4-96156317facb admin admin] Request body: {'subnet': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'enable_dhcp': False, 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'name': 'public-subnet', 'ip_version': 4}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:14:52.839404 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-9c6d9180-1c3b-47b3-99b4-96156317facb admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:14:52.843272 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-9c6d9180-1c3b-47b3-99b4-96156317facb admin admin] Usage tracker for resource:subnet and tenant:2361ee7db41742a7b41efa29b36d3870 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:14:52.846554 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-9c6d9180-1c3b-47b3-99b4-96156317facb admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:14:52.852291 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-9c6d9180-1c3b-47b3-99b4-96156317facb admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:14:52.863684 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-9c6d9180-1c3b-47b3-99b4-96156317facb admin admin] Made reservation on behalf of 2361ee7db41742a7b41efa29b36d3870 for: {'subnet': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:14:52.865947 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9c6d9180-1c3b-47b3-99b4-96156317facb admin admin] Notify callbacks [] for subnet, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:53.285967 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.objects.ports [None req-9c6d9180-1c3b-47b3-99b4-96156317facb admin admin] Time-cost: Fetching 0 ports in 0 security groups {{(pid=91643) get_objects /opt/stack/neutron/neutron/objects/ports.py:443}} May 20 04:14:53.286402 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9c6d9180-1c3b-47b3-99b4-96156317facb admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868'] for subnet, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:53.289814 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:14:53.355986 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9c6d9180-1c3b-47b3-99b4-96156317facb admin admin] Pushing event updated for resources: {'Subnet': ['ID=e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:14:53.361212 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9c6d9180-1c3b-47b3-99b4-96156317facb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:14:53.365424 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9c6d9180-1c3b-47b3-99b4-96156317facb admin admin] Checking segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91643) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:14:53.365769 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-9c6d9180-1c3b-47b3-99b4-96156317facb admin admin] Network hostable DHCP agents. Network: 712ce8c4-8feb-4719-bf13-e06992125cd9, hostable agents: ['fffae25a-f450-4182-8f9e-ac29d143f6c9'], hosted agents: [] {{(pid=91643) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 20 04:14:53.385178 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-9c6d9180-1c3b-47b3-99b4-96156317facb admin admin] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 is scheduled to be hosted by DHCP agent fffae25a-f450-4182-8f9e-ac29d143f6c9 with binding_index 1 {{(pid=91643) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 20 04:14:53.395381 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-9c6d9180-1c3b-47b3-99b4-96156317facb admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-782405-9-iscsi, 712ce8c4-8feb-4719-bf13-e06992125cd9) {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:14:53.399313 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-9c6d9180-1c3b-47b3-99b4-96156317facb admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:14:53.399880 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9c6d9180-1c3b-47b3-99b4-96156317facb admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:53.404972 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-48075fdb-6ea0-489f-bc9b-67a1a9d799fb None None] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:14:53.431252 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-9c6d9180-1c3b-47b3-99b4-96156317facb admin admin] 10.222.0.39 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 821 time: 0.5892937 May 20 04:14:53.606436 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-ff8e192c-28c2-4613-9691-ed9896b87166 None None] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:14:55.620389 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 59452) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:14:55.835102 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-630d198f-f95e-45a0-a195-6cf93e0203fa admin admin] 10.222.0.39 "GET /v2.0/routers/ae96cfaf-58ff-48ee-bb34-133e5546ed99 HTTP/1.1" status: 200 len: 710 time: 0.2135911 May 20 04:14:55.973820 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-b5fb4419-89ad-4e06-8623-8535b19e7032 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:14:55.974297 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-b5fb4419-89ad-4e06-8623-8535b19e7032 admin admin] Synchronizing usage tracker for tenant:2361ee7db41742a7b41efa29b36d3870 on resource:network {{(pid=91644) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:14:55.992360 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-b5fb4419-89ad-4e06-8623-8535b19e7032 admin admin] Unset dirty status for tenant:2361ee7db41742a7b41efa29b36d3870 on resource:network {{(pid=91644) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:14:55.993614 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-b5fb4419-89ad-4e06-8623-8535b19e7032 admin admin] 10.222.0.39 "GET /v2.0/networks/712ce8c4-8feb-4719-bf13-e06992125cd9 HTTP/1.1" status: 200 len: 886 time: 0.1531701 May 20 04:14:56.002327 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-50ff10a4-0e23-47ab-b8dd-9b1450579cf2 admin admin] Request body: {'router': {'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:14:56.452787 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-50ff10a4-0e23-47ab-b8dd-9b1450579cf2 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-8053308', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-8071430', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-260685'] for router, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:56.821080 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-50ff10a4-0e23-47ab-b8dd-9b1450579cf2 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:56.843441 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device ab0a3324-2266-4f95-bb12-ee62475ae67c up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:14:56.879790 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-50ff10a4-0e23-47ab-b8dd-9b1450579cf2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:56.911639 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port ab0a3324-2266-4f95-bb12-ee62475ae67c completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:14:56.916303 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port ab0a3324-2266-4f95-bb12-ee62475ae67c triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:14:56.916676 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:57.013891 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [None req-50ff10a4-0e23-47ab-b8dd-9b1450579cf2 admin admin] Allocated IP 172.27.0.76 (712ce8c4-8feb-4719-bf13-e06992125cd9/e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5/dc2e6b2c-5457-459c-8db2-920aa66b51c0) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:14:57.032503 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:57.065013 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port ab0a3324-2266-4f95-bb12-ee62475ae67c, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=ab0a3324-2266-4f95-bb12-ee62475ae67c,segment=NetworkSegment(6879b57a-f9a6-4d4e-b5a0-dc24c6cb79e2),segment_id=6879b57a-f9a6-4d4e-b5a0-dc24c6cb79e2)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:14:57.069988 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:57.072047 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:14:57.139843 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=ab0a3324-2266-4f95-bb12-ee62475ae67c,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:14:57.143169 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:14:57.162298 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-50ff10a4-0e23-47ab-b8dd-9b1450579cf2 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:57.222140 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-50ff10a4-0e23-47ab-b8dd-9b1450579cf2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:57.225859 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:14:57.285727 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-50ff10a4-0e23-47ab-b8dd-9b1450579cf2 admin admin] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:14:57.288324 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-50ff10a4-0e23-47ab-b8dd-9b1450579cf2 admin admin] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:14:57.291787 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-50ff10a4-0e23-47ab-b8dd-9b1450579cf2 admin admin] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:14:57.346815 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-50ff10a4-0e23-47ab-b8dd-9b1450579cf2 admin admin] Pushing event updated for resources: {'Port': ['ID=dc2e6b2c-5457-459c-8db2-920aa66b51c0,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:14:57.357575 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-50ff10a4-0e23-47ab-b8dd-9b1450579cf2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.132s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:14:57.671678 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-50ff10a4-0e23-47ab-b8dd-9b1450579cf2 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:58.050706 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-50ff10a4-0e23-47ab-b8dd-9b1450579cf2 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-8053308', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-8071430', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-260685'] for router, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:58.054141 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-50ff10a4-0e23-47ab-b8dd-9b1450579cf2 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-8053391', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_distributed_port_bindings_after_change-8053501', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_snat_interfaces_after_change-8053380', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._reconfigure_ha_resources-8071448'] for router, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:58.256715 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-50ff10a4-0e23-47ab-b8dd-9b1450579cf2 admin admin] Router ae96cfaf-58ff-48ee-bb34-133e5546ed99 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91644) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:14:58.433259 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-50ff10a4-0e23-47ab-b8dd-9b1450579cf2 admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:14:58.441589 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-50ff10a4-0e23-47ab-b8dd-9b1450579cf2 admin admin] Notify callbacks [] for router, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:58.442755 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-50ff10a4-0e23-47ab-b8dd-9b1450579cf2 admin admin] 10.222.0.39 "PUT /v2.0/routers/ae96cfaf-58ff-48ee-bb34-133e5546ed99 HTTP/1.1" status: 200 len: 889 time: 2.4437661 May 20 04:14:58.443464 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Sync routers for ids ['ae96cfaf-58ff-48ee-bb34-133e5546ed99'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:14:58.454087 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Routers {'ae96cfaf-58ff-48ee-bb34-133e5546ed99'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:14:58.454759 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 0, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:14:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['ae96cfaf-58ff-48ee-bb34-133e5546ed99'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:14:58.647598 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Set router ae96cfaf-58ff-48ee-bb34-133e5546ed99 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:14:58.882719 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device ab0a3324-2266-4f95-bb12-ee62475ae67c up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:14:59.021110 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port ab0a3324-2266-4f95-bb12-ee62475ae67c triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:14:59.021403 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:59.164433 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'ae96cfaf-58ff-48ee-bb34-133e5546ed99', 'name': 'router1', 'tenant_id': '61e4cb63ed58450b84b65106d258ea76', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': 'dc2e6b2c-5457-459c-8db2-920aa66b51c0', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:14:39Z', 'updated_at': '2021-05-20T04:14:57Z', 'revision_number': 4, 'project_id': '61e4cb63ed58450b84b65106d258ea76', 'gw_port': {'id': 'dc2e6b2c-5457-459c-8db2-920aa66b51c0', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:37:29:92', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'ae96cfaf-58ff-48ee-bb34-133e5546ed99', 'device_owner': 'network:router_gateway', 'standard_attr_id': 21, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:14:56Z', 'updated_at': '2021-05-20T04:14:57Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'ab0a3324-2266-4f95-bb12-ee62475ae67c', 'name': '', 'network_id': '65c1116d-8a94-470d-8f12-ff3ae63f0c9d', 'tenant_id': '61e4cb63ed58450b84b65106d258ea76', 'mac_address': 'fa:16:3e:b3:e0:82', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ae96cfaf-58ff-48ee-bb34-133e5546ed99', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '10045323-b78d-4e34-b0af-361098232ecf', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:14:48Z', 'updated_at': '2021-05-20T04:14:57Z', 'revision_number': 5, 'project_id': '61e4cb63ed58450b84b65106d258ea76', 'subnets': [{'id': '10045323-b78d-4e34-b0af-361098232ecf', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': 'b8e14113-efb9-487e-a28c-298285e213da'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:14:59.169574 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Checking router: ae96cfaf-58ff-48ee-bb34-133e5546ed99 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:14:59.169924 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Updating router ae96cfaf-58ff-48ee-bb34-133e5546ed99 port dc2e6b2c-5457-459c-8db2-920aa66b51c0 binding host dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 20 04:14:59.221021 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:59.550319 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Notify callbacks [] for port, precommit_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:59.562421 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Transition to ACTIVE for port object dc2e6b2c-5457-459c-8db2-920aa66b51c0 will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:14:59.569284 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:59.573869 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:14:59.649490 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Attempting to bind port dc2e6b2c-5457-459c-8db2-920aa66b51c0 on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:14:59.649857 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Attempting to bind port dc2e6b2c-5457-459c-8db2-920aa66b51c0 by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:14:59.650216 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Attempting to bind port dc2e6b2c-5457-459c-8db2-920aa66b51c0 on network 712ce8c4-8feb-4719-bf13-e06992125cd9 {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:14:59.655696 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fddf7d60-b533-4f61-be94-88c30852cf58 None None] Pushing event updated for resources: {'Port': ['ID=dc2e6b2c-5457-459c-8db2-920aa66b51c0,revision_number=2']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:14:59.659451 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-fddf7d60-b533-4f61-be94-88c30852cf58 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:14:59.660336 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 14, 45), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:14:59.660699 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Checking segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:14:59.661093 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:59.661463 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Bound using segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:14:59.661924 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Bound port: dc2e6b2c-5457-459c-8db2-920aa66b51c0, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:14:59.662288 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:59.733220 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] For port dc2e6b2c-5457-459c-8db2-920aa66b51c0, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:14:59.736657 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] For port dc2e6b2c-5457-459c-8db2-920aa66b51c0, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=dc2e6b2c-5457-459c-8db2-920aa66b51c0,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:14:59.741626 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:14:59.745525 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-dddd9bb9-97b5-40b8-8347-e455b4eaaea2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:14:59.821040 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fddf7d60-b533-4f61-be94-88c30852cf58 None None] Pushing event updated for resources: {'Port': ['ID=dc2e6b2c-5457-459c-8db2-920aa66b51c0,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:14:59.823486 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Routers {'ae96cfaf-58ff-48ee-bb34-133e5546ed99'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:14:59.824074 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 0, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:14:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['ae96cfaf-58ff-48ee-bb34-133e5546ed99'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:14:59.870063 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-fddf7d60-b533-4f61-be94-88c30852cf58 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.124s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:15:00.000930 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] Set router ae96cfaf-58ff-48ee-bb34-133e5546ed99 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:15:00.354408 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'ae96cfaf-58ff-48ee-bb34-133e5546ed99', 'name': 'router1', 'tenant_id': '61e4cb63ed58450b84b65106d258ea76', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': 'dc2e6b2c-5457-459c-8db2-920aa66b51c0', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:14:39Z', 'updated_at': '2021-05-20T04:14:57Z', 'revision_number': 4, 'project_id': '61e4cb63ed58450b84b65106d258ea76', 'gw_port': {'id': 'dc2e6b2c-5457-459c-8db2-920aa66b51c0', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:37:29:92', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'ae96cfaf-58ff-48ee-bb34-133e5546ed99', 'device_owner': 'network:router_gateway', 'standard_attr_id': 21, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:14:56Z', 'updated_at': '2021-05-20T04:14:59Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'ab0a3324-2266-4f95-bb12-ee62475ae67c', 'name': '', 'network_id': '65c1116d-8a94-470d-8f12-ff3ae63f0c9d', 'tenant_id': '61e4cb63ed58450b84b65106d258ea76', 'mac_address': 'fa:16:3e:b3:e0:82', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ae96cfaf-58ff-48ee-bb34-133e5546ed99', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '10045323-b78d-4e34-b0af-361098232ecf', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:14:48Z', 'updated_at': '2021-05-20T04:14:57Z', 'revision_number': 5, 'project_id': '61e4cb63ed58450b84b65106d258ea76', 'subnets': [{'id': '10045323-b78d-4e34-b0af-361098232ecf', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': 'b8e14113-efb9-487e-a28c-298285e213da'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:15:00.358818 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0069d5d2-5c79-4862-bdcd-610af0a62bbb None None] The sync data for ids ['ae96cfaf-58ff-48ee-bb34-133e5546ed99'] in dv-cinder-782405-9-iscsi is: [{'id': 'ae96cfaf-58ff-48ee-bb34-133e5546ed99', 'name': 'router1', 'tenant_id': '61e4cb63ed58450b84b65106d258ea76', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.76'}], 'enable_snat': True}, 'gw_port_id': 'dc2e6b2c-5457-459c-8db2-920aa66b51c0', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:14:39Z', 'updated_at': '2021-05-20T04:14:57Z', 'revision_number': 4, 'project_id': '61e4cb63ed58450b84b65106d258ea76', 'gw_port': {'id': 'dc2e6b2c-5457-459c-8db2-920aa66b51c0', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:37:29:92', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'ae96cfaf-58ff-48ee-bb34-133e5546ed99', 'device_owner': 'network:router_gateway', 'standard_attr_id': 21, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.76', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:14:56Z', 'updated_at': '2021-05-20T04:14:59Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'ab0a3324-2266-4f95-bb12-ee62475ae67c', 'name': '', 'network_id': '65c1116d-8a94-470d-8f12-ff3ae63f0c9d', 'tenant_id': '61e4cb63ed58450b84b65106d258ea76', 'mac_address': 'fa:16:3e:b3:e0:82', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ae96cfaf-58ff-48ee-bb34-133e5546ed99', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '10045323-b78d-4e34-b0af-361098232ecf', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:14:48Z', 'updated_at': '2021-05-20T04:14:57Z', 'revision_number': 5, 'project_id': '61e4cb63ed58450b84b65106d258ea76', 'subnets': [{'id': '10045323-b78d-4e34-b0af-361098232ecf', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': 'b8e14113-efb9-487e-a28c-298285e213da'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:15:00.741037 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 59462) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:15:00.861755 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device dc2e6b2c-5457-459c-8db2-920aa66b51c0 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:15:00.863849 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-2bdd2b9c-140e-43d0-b556-937557282035 admin admin] 10.222.0.39 "GET /v2.0/ports?device_owner=network%3Arouter_gateway&fields=id&fields=name&fields=mac_address&fields=fixed_ips&fields=status HTTP/1.1" status: 200 len: 410 time: 0.1216624 May 20 04:15:00.939781 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port dc2e6b2c-5457-459c-8db2-920aa66b51c0 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:15:00.944890 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port dc2e6b2c-5457-459c-8db2-920aa66b51c0 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:15:00.945261 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:15:01.074552 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:15:01.093441 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port dc2e6b2c-5457-459c-8db2-920aa66b51c0, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=dc2e6b2c-5457-459c-8db2-920aa66b51c0,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:15:01.098262 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:15:01.101397 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-fddf7d60-b533-4f61-be94-88c30852cf58 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:15:01.168335 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=dc2e6b2c-5457-459c-8db2-920aa66b51c0,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:15:01.172241 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:15:02.847450 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device dc2e6b2c-5457-459c-8db2-920aa66b51c0 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:15:02.920271 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port dc2e6b2c-5457-459c-8db2-920aa66b51c0 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:15:02.920956 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:15:12.587110 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-85306a61-fb6c-4076-926b-b919dabe37c0 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:15:14.943042 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:15:15.067524 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-272868c2-ad21-47d8-addd-bdae5d1b3b10 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:15:15.782443 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 5 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:15:16.113730 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:15:17.749311 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:15:42.583157 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-2695eed3-c1b4-4bf5-acd8-71d953832166 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:15:44.931283 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:15:45.051492 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c1ed37c1-f136-4da1-9010-8e3407472ba3 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:15:46.116199 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:15:47.745231 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:15:52.782447 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 5 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:16:12.577394 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-98ec07e2-6cfd-4635-a02c-dcd5cb9eab02 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:16:14.936956 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:16:15.054502 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-fff8bd0a-9ed8-41d7-8f19-b93cc498485e None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:16:16.119867 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:16:17.748796 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:16:29.782684 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 5 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:16:34.289107 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 60032) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:16:34.372270 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.pecan_wsgi.hooks.translation [None req-73b49e62-f961-4dfc-b7c4-84ac4d864254 admin admin] GET failed (client error): The resource could not be found. May 20 04:16:34.373829 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-73b49e62-f961-4dfc-b7c4-84ac4d864254 admin admin] 10.222.0.39 "GET /v2.0/networks/private HTTP/1.1" status: 404 len: 315 time: 0.0837395 May 20 04:16:34.510484 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-6909401e-546d-4131-babc-665f59c2bc42 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:16:34.511947 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-6909401e-546d-4131-babc-665f59c2bc42 admin admin] 10.222.0.39 "GET /v2.0/networks?name=private HTTP/1.1" status: 200 len: 872 time: 0.1322973 May 20 04:16:36.596077 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 60042) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:16:36.715621 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-9e874070-2a2a-49d0-851c-e25b4951752b admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:16:36.717002 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-9e874070-2a2a-49d0-851c-e25b4951752b admin admin] 10.222.0.39 "GET /v2.0/subnets/10045323-b78d-4e34-b0af-361098232ecf HTTP/1.1" status: 200 len: 849 time: 0.1195548 May 20 04:16:36.729512 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] Request body: {'subnet': {'dns_nameservers': ['8.8.8.8', '8.8.4.4']}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:16:36.845616 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] Notify callbacks [] for subnet, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:16:37.033833 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_subnet_gateway_ip_update-8760688930982'] for subnet, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:16:37.041134 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-0ed6c66c-06fc-4a42-9597-8b49eeea0d47 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:16:37.078149 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:16:37.078730 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:16:37.080012 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] 10.222.0.39 "PUT /v2.0/subnets/10045323-b78d-4e34-b0af-361098232ecf HTTP/1.1" status: 200 len: 868 time: 0.3542559 May 20 04:16:37.098635 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-c3fc12cd-1a8c-4354-a044-96d86234a13b None None] Network 65c1116d-8a94-470d-8f12-ff3ae63f0c9d requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:16:37.112303 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] Pushing event updated for resources: {'Subnet': ['ID=10045323-b78d-4e34-b0af-361098232ecf,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:16:37.120056 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:16:37.120447 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:16:37.120733 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:16:37.121076 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:16:37.121390 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:16:37.121731 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:16:37.122035 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:16:37.122351 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:16:37.122697 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:16:37.123063 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:16:37.123407 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:16:37.127045 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:16:37.130095 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.089s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:16:37.528839 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-9a341373-cafb-4502-9f37-8244fb010839 None None] Provisioning complete for port 03798bed-9045-4ff6-8d71-a4df490fc833 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:16:37.529306 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-9a341373-cafb-4502-9f37-8244fb010839 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:16:37.653567 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-9a341373-cafb-4502-9f37-8244fb010839 None None] Provisioning complete for port ab0a3324-2266-4f95-bb12-ee62475ae67c triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:16:37.653840 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-9a341373-cafb-4502-9f37-8244fb010839 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:16:39.184984 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60052) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:16:39.423502 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-cb4ac053-3caf-4405-8734-2c80c38fedf7 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:16:39.424089 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-cb4ac053-3caf-4405-8734-2c80c38fedf7 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:16:39.426300 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-cb4ac053-3caf-4405-8734-2c80c38fedf7 admin admin] 10.222.0.39 "GET /v2.0/networks HTTP/1.1" status: 200 len: 1549 time: 0.2402868 May 20 04:16:41.556689 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60062) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:16:41.738283 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-cf9622f1-7891-46c2-b489-25a785150b4c admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:16:41.738676 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-cf9622f1-7891-46c2-b489-25a785150b4c admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:16:41.740222 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-cf9622f1-7891-46c2-b489-25a785150b4c admin admin] 10.222.0.39 "GET /v2.0/networks HTTP/1.1" status: 200 len: 1549 time: 0.1825430 May 20 04:16:42.581283 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-b9f00a1b-9a6c-436a-b193-a4270fd3158a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:16:43.819835 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60072) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:16:43.997325 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-0ef81366-2ad5-470b-aa6f-75cf66eb5876 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:16:43.999016 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-0ef81366-2ad5-470b-aa6f-75cf66eb5876 admin admin] 10.222.0.39 "GET /v2.0/networks/65c1116d-8a94-470d-8f12-ff3ae63f0c9d HTTP/1.1" status: 200 len: 869 time: 0.1780496 May 20 04:16:44.941478 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:16:45.067926 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-86dc133e-4fc4-427f-a648-567e91aaf203 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:16:46.120865 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 60082) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:16:46.123597 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:16:46.289945 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-2437a5dc-9d69-4f31-bf88-1cda5a1a50d8 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:16:46.291412 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-2437a5dc-9d69-4f31-bf88-1cda5a1a50d8 admin admin] 10.222.0.39 "GET /v2.0/networks/712ce8c4-8feb-4719-bf13-e06992125cd9 HTTP/1.1" status: 200 len: 886 time: 0.1695070 May 20 04:16:47.752231 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:16:48.374038 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 60092) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:16:48.502539 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-9449da68-90a5-46d2-9148-e21bab95698d admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:16:48.502999 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-9449da68-90a5-46d2-9148-e21bab95698d admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:16:48.504537 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-9449da68-90a5-46d2-9148-e21bab95698d admin admin] 10.222.0.39 "GET /v2.0/subnets HTTP/1.1" status: 200 len: 1479 time: 0.1293166 May 20 04:16:50.612851 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60102) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:16:50.723180 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-a60734b3-0774-4659-98e1-9bb7ff5bcefd admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:16:50.723663 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-a60734b3-0774-4659-98e1-9bb7ff5bcefd admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:16:50.724378 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-a60734b3-0774-4659-98e1-9bb7ff5bcefd admin admin] Synchronizing usage tracker for tenant:2361ee7db41742a7b41efa29b36d3870 on resource:subnet {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:16:50.743623 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-a60734b3-0774-4659-98e1-9bb7ff5bcefd admin admin] Unset dirty status for tenant:2361ee7db41742a7b41efa29b36d3870 on resource:subnet {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:16:50.745155 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-a60734b3-0774-4659-98e1-9bb7ff5bcefd admin admin] 10.222.0.39 "GET /v2.0/subnets HTTP/1.1" status: 200 len: 1479 time: 0.1313744 May 20 04:16:52.821386 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60112) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:16:52.933224 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-0d3ebec2-7773-4beb-ac04-4063faaca77c admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:16:52.934510 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-0d3ebec2-7773-4beb-ac04-4063faaca77c admin admin] 10.222.0.39 "GET /v2.0/subnets/10045323-b78d-4e34-b0af-361098232ecf HTTP/1.1" status: 200 len: 868 time: 0.1121807 May 20 04:16:55.041013 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60122) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:16:55.152890 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-1626801d-187a-4598-8c76-bf7fc2b68029 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:16:55.154123 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-1626801d-187a-4598-8c76-bf7fc2b68029 admin admin] 10.222.0.39 "GET /v2.0/subnets/e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5 HTTP/1.1" status: 200 len: 816 time: 0.1121197 May 20 04:17:06.782391 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 5 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:17:12.586470 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-ba4d465f-299c-4a35-ac53-e02993ef4f2b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:14.945586 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:15.061526 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-97a839f3-2c0a-45a1-be12-dc6853a5db5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:16.123037 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:17.756394 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:17.975692 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 60166) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:17:18.024522 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [None req-41bd4742-1719-4e76-85a6-1d1b4c79f52a admin admin] http://10.222.0.39:9696/v2.0/extensions returned with HTTP 200 {{(pid=91644) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 20 04:17:18.025487 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-41bd4742-1719-4e76-85a6-1d1b4c79f52a admin admin] 10.222.0.39 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0488765 May 20 04:17:20.211926 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 60176) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:17:20.284718 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.pecan_wsgi.hooks.translation [None req-a1b25b72-cfd9-4749-b490-85a058a2ffe5 admin admin] GET failed (client error): The resource could not be found. May 20 04:17:20.285926 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-a1b25b72-cfd9-4749-b490-85a058a2ffe5 admin admin] 10.222.0.39 "GET /v2.0/networks/public HTTP/1.1" status: 404 len: 314 time: 0.0731139 May 20 04:17:20.420488 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-ded5abac-bb55-40e2-97e8-c26b9e9d08e9 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:17:20.421827 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-ded5abac-bb55-40e2-97e8-c26b9e9d08e9 admin admin] 10.222.0.39 "GET /v2.0/networks?name=public HTTP/1.1" status: 200 len: 889 time: 0.1299140 May 20 04:17:22.542568 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60186) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:17:22.595615 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Request body: {'network': {'admin_state_up': True, 'shared': True, 'name': 'shared'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:17:22.600417 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:17:22.610685 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Attempting to reserve 1 items for resource network. Total usage: 1; quota limit: 100; headroom:99 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:17:22.623631 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Made reservation on behalf of 2361ee7db41742a7b41efa29b36d3870 for: {'network': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:17:22.625932 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-585852'] for network, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:22.711816 dv-cinder-782405-9-iscsi neutron-server[91643]: WARNING neutron.services.revisions.revision_plugin [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Could not find related network for resource to bump revision. May 20 04:17:22.725477 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 816} {{(pid=91643) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 20 04:17:22.734417 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:22.734759 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.segments_db [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Added segment 0fbdfc2a-8204-4d65-bf3c-9044da0482d3 of type vlan for network 504e252b-5361-4c2a-87b4-bd9946954761 May 20 04:17:22.759455 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8760688516720'] for network, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:22.790841 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915'] for network, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:22.792419 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:17:22.793005 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:22.797574 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:17:22.890995 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] 10.222.0.39 "POST /v2.0/networks HTTP/1.1" status: 201 len: 847 time: 0.3472192 May 20 04:17:22.895920 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Pushing event updated for resources: {'Network': ['ID=504e252b-5361-4c2a-87b4-bd9946954761,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:17:22.904089 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:22.904475 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:22.904813 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:22.905143 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:22.905462 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:22.905823 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:22.906171 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:22.906508 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:22.906847 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:22.907220 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:22.907612 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:22.908093 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:22.912715 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.115s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:17:22.919749 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Network 504e252b-5361-4c2a-87b4-bd9946954761 details requested by agent ovs-agent-dv-cinder-782405-9-iscsi with host dv-cinder-782405-9-iscsi {{(pid=91645) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 20 04:17:24.988228 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60196) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:17:25.057269 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.pecan_wsgi.hooks.translation [None req-cfe89023-933a-40f3-8af8-c2ee59c718f0 admin admin] GET failed (client error): The resource could not be found. May 20 04:17:25.057787 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-cfe89023-933a-40f3-8af8-c2ee59c718f0 admin admin] Synchronizing usage tracker for tenant:2361ee7db41742a7b41efa29b36d3870 on resource:network {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:17:25.077439 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-cfe89023-933a-40f3-8af8-c2ee59c718f0 admin admin] Unset dirty status for tenant:2361ee7db41742a7b41efa29b36d3870 on resource:network {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:17:25.078744 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-cfe89023-933a-40f3-8af8-c2ee59c718f0 admin admin] 10.222.0.39 "GET /v2.0/networks/shared HTTP/1.1" status: 404 len: 314 time: 0.0895488 May 20 04:17:25.191996 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-d771ee7d-47fa-4d13-9087-a3da5f6f7844 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:17:25.193331 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-d771ee7d-47fa-4d13-9087-a3da5f6f7844 admin admin] 10.222.0.39 "GET /v2.0/networks?name=shared HTTP/1.1" status: 200 len: 826 time: 0.1105444 May 20 04:17:25.205396 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-c07f55d5-5609-454b-971f-f74a83d9d176 admin admin] Request body: {'subnet': {'network_id': '504e252b-5361-4c2a-87b4-bd9946954761', 'cidr': '192.168.233.0/24', 'ip_version': 4, 'name': 'shared-subnet', 'description': 'shared-subnet'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:17:25.209798 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-c07f55d5-5609-454b-971f-f74a83d9d176 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:17:25.219635 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-c07f55d5-5609-454b-971f-f74a83d9d176 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 1; quota limit: 100; headroom:99 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:17:25.232960 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-c07f55d5-5609-454b-971f-f74a83d9d176 admin admin] Made reservation on behalf of 2361ee7db41742a7b41efa29b36d3870 for: {'subnet': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:17:25.235161 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c07f55d5-5609-454b-971f-f74a83d9d176 admin admin] Notify callbacks [] for subnet, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:25.626735 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c07f55d5-5609-454b-971f-f74a83d9d176 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868'] for subnet, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:25.630131 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:17:25.695730 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c07f55d5-5609-454b-971f-f74a83d9d176 admin admin] Pushing event updated for resources: {'Subnet': ['ID=3d544974-6fec-477e-a946-f434a49fa867,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:17:25.699365 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-c07f55d5-5609-454b-971f-f74a83d9d176 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:17:25.706228 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c07f55d5-5609-454b-971f-f74a83d9d176 admin admin] Checking segment: {'id': '0fbdfc2a-8204-4d65-bf3c-9044da0482d3', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 816, 'network_id': '504e252b-5361-4c2a-87b4-bd9946954761'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91643) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:17:25.706571 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c07f55d5-5609-454b-971f-f74a83d9d176 admin admin] Network hostable DHCP agents. Network: 504e252b-5361-4c2a-87b4-bd9946954761, hostable agents: ['fffae25a-f450-4182-8f9e-ac29d143f6c9'], hosted agents: [] {{(pid=91643) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 20 04:17:25.725600 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c07f55d5-5609-454b-971f-f74a83d9d176 admin admin] Network 504e252b-5361-4c2a-87b4-bd9946954761 is scheduled to be hosted by DHCP agent fffae25a-f450-4182-8f9e-ac29d143f6c9 with binding_index 1 {{(pid=91643) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 20 04:17:25.735975 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-c07f55d5-5609-454b-971f-f74a83d9d176 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-782405-9-iscsi, 504e252b-5361-4c2a-87b4-bd9946954761) {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:17:25.740128 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-c07f55d5-5609-454b-971f-f74a83d9d176 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:17:25.740703 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c07f55d5-5609-454b-971f-f74a83d9d176 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:25.745301 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-bb96935a-68e5-4bca-8ae8-662ced8e3db3 None None] Network 504e252b-5361-4c2a-87b4-bd9946954761 requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:17:25.815665 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-c07f55d5-5609-454b-971f-f74a83d9d176 admin admin] 10.222.0.39 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 845 time: 0.6133087 May 20 04:17:25.999049 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcp85c0b5c8-c799-5455-967d-3e9c2c8b4839-504e252b-5361-4c2a-87b4-bd9946954761', 'network_id': '504e252b-5361-4c2a-87b4-bd9946954761', 'tenant_id': '2361ee7db41742a7b41efa29b36d3870', 'fixed_ips': [{'subnet_id': '3d544974-6fec-477e-a946-f434a49fa867'}]}} from dv-cinder-782405-9-iscsi. {{(pid=91645) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 20 04:17:25.999922 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:26.155629 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.db_base_plugin_common [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Allocated IP 192.168.233.2 (504e252b-5361-4c2a-87b4-bd9946954761/3d544974-6fec-477e-a946-f434a49fa867/2323fd63-3af4-4896-a38b-fe882c2aca15) {{(pid=91645) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:17:26.360243 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Notify callbacks [] for port, precommit_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:26.371071 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Transition to ACTIVE for port object 2323fd63-3af4-4896-a38b-fe882c2aca15 will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:17:26.390213 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Transition to ACTIVE for port object 2323fd63-3af4-4896-a38b-fe882c2aca15 will not be triggered until provisioned by entity DHCP. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:17:26.396905 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:26.401897 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-337d9a46-1a14-4b24-94dc-a0c6dceddf4f None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:17:26.464311 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Network 504e252b-5361-4c2a-87b4-bd9946954761 is already hosted by enough agents. {{(pid=91645) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:17:26.467326 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.base_scheduler [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Resources bound (agent type, host, resource id): {{(pid=91645) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:17:26.470146 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Received port after_create {{(pid=91645) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:17:26.503715 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Pushing event updated for resources: {'Port': ['ID=2323fd63-3af4-4896-a38b-fe882c2aca15,revision_number=1']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:17:26.518859 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:26.519170 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:26.519554 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:26.519858 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:26.520179 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:26.520500 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:26.520801 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:26.521118 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:26.521450 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:26.521764 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:26.522076 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:26.522387 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:17:26.526111 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.124s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:17:26.578134 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Attempting to bind port 2323fd63-3af4-4896-a38b-fe882c2aca15 on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:17:26.578484 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Attempting to bind port 2323fd63-3af4-4896-a38b-fe882c2aca15 by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '0fbdfc2a-8204-4d65-bf3c-9044da0482d3', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 816, 'network_id': '504e252b-5361-4c2a-87b4-bd9946954761'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:17:26.578858 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Attempting to bind port 2323fd63-3af4-4896-a38b-fe882c2aca15 on network 504e252b-5361-4c2a-87b4-bd9946954761 {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:17:26.584249 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 17, 15), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:17:26.584617 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Checking segment: {'id': '0fbdfc2a-8204-4d65-bf3c-9044da0482d3', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 816, 'network_id': '504e252b-5361-4c2a-87b4-bd9946954761'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:17:26.585013 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:26.585411 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Bound using segment: {'id': '0fbdfc2a-8204-4d65-bf3c-9044da0482d3', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 816, 'network_id': '504e252b-5361-4c2a-87b4-bd9946954761'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:17:26.585870 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Bound port: 2323fd63-3af4-4896-a38b-fe882c2aca15, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '0fbdfc2a-8204-4d65-bf3c-9044da0482d3', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 816, 'network_id': '504e252b-5361-4c2a-87b4-bd9946954761'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:17:26.586231 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:26.671453 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] For port 2323fd63-3af4-4896-a38b-fe882c2aca15, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:17:26.674535 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] For port 2323fd63-3af4-4896-a38b-fe882c2aca15, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=2323fd63-3af4-4896-a38b-fe882c2aca15,segment=NetworkSegment(0fbdfc2a-8204-4d65-bf3c-9044da0482d3),segment_id=0fbdfc2a-8204-4d65-bf3c-9044da0482d3)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:17:26.678607 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:26.682373 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:17:26.758143 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Pushing event updated for resources: {'Port': ['ID=2323fd63-3af4-4896-a38b-fe882c2aca15,revision_number=2']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:17:26.760910 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:17:27.548865 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-fe76b5a8-8386-4f33-b850-a0ac388c438f None None] Network 504e252b-5361-4c2a-87b4-bd9946954761 requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:17:27.652302 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-ada11e9f-5f38-4532-b55e-468d8a151654 None None] Provisioning for port 2323fd63-3af4-4896-a38b-fe882c2aca15 completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:17:28.955443 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 2323fd63-3af4-4896-a38b-fe882c2aca15 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:17:29.029039 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port 2323fd63-3af4-4896-a38b-fe882c2aca15 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:17:29.033995 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 2323fd63-3af4-4896-a38b-fe882c2aca15 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:17:29.034369 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:29.152692 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:29.185784 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port 2323fd63-3af4-4896-a38b-fe882c2aca15, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=2323fd63-3af4-4896-a38b-fe882c2aca15,segment=NetworkSegment(0fbdfc2a-8204-4d65-bf3c-9044da0482d3),segment_id=0fbdfc2a-8204-4d65-bf3c-9044da0482d3)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:17:29.191160 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:29.193411 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:17:29.259446 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=2323fd63-3af4-4896-a38b-fe882c2aca15,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:17:29.262575 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:17:30.935180 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 2323fd63-3af4-4896-a38b-fe882c2aca15 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:17:31.008884 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 2323fd63-3af4-4896-a38b-fe882c2aca15 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:17:31.009250 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:42.580381 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-8f04e69c-131d-45a7-bb76-893c18f9469d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:43.782425 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 5 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:17:44.953592 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:45.070815 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-5ebc7627-2c8b-4a1f-90a0-12b467c28025 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:46.126762 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:17:47.757932 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:18:12.586124 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-98d0507c-85e9-4381-be32-4444925ae523 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:18:14.949044 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:18:15.066390 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-805ae21a-e1eb-4d63-abdc-20b57dbd4919 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:18:16.129542 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:18:17.761685 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:18:20.781755 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 5 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:18:30.908813 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60246) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:18:30.915351 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [None req-1f420ecc-ca83-4629-9573-71f75b5ad7b8 tempest-verify_tempest_config-691916556 tempest-verify_tempest_config-691916556-project] http://10.222.0.39:9696/v2.0/extensions returned with HTTP 200 {{(pid=91643) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 20 04:18:30.916203 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-1f420ecc-ca83-4629-9573-71f75b5ad7b8 tempest-verify_tempest_config-691916556 tempest-verify_tempest_config-691916556-project] 10.222.0.39 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13635 time: 0.0063241 May 20 04:18:31.243814 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 60264) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:18:31.378963 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:18:31.394209 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:18:31.398520 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Usage tracker for resource:security_group_rule and tenant:05197aefb4c845089450af2d74c7dcfc is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:18:31.401888 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:18:31.407877 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:18:31.545776 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:18:31.551631 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:18:31.558802 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:18:31.596721 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f77bdcde-6735-49c1-82e1-4b2936088649,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:18:31.599673 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Synchronizing usage tracker for tenant:2361ee7db41742a7b41efa29b36d3870 on resource:security_group {{(pid=91644) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:18:31.607840 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:18:31.608174 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:18:31.608499 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:18:31.608815 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:18:31.609164 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:18:31.609476 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:18:31.609783 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:18:31.610089 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:18:31.610414 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:18:31.610703 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:18:31.611007 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:18:31.611333 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:18:31.616217 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.057s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:18:31.628982 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Unset dirty status for tenant:2361ee7db41742a7b41efa29b36d3870 on resource:security_group {{(pid=91644) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:18:31.630275 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=05197aefb4c845089450af2d74c7dcfc&name=default HTTP/1.1" status: 200 len: 2975 time: 0.3854418 May 20 04:18:31.633220 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 60270) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:18:31.690587 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-71933fe8-dfea-47b9-b610-dcfe642e1cc4 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:18:31.715749 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-71933fe8-dfea-47b9-b610-dcfe642e1cc4 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:18:31.818106 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-71933fe8-dfea-47b9-b610-dcfe642e1cc4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:18:31.822134 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:18:31.833098 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-71933fe8-dfea-47b9-b610-dcfe642e1cc4 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f77bdcde-6735-49c1-82e1-4b2936088649,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:18:31.835821 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-71933fe8-dfea-47b9-b610-dcfe642e1cc4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.013s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:18:31.841268 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-71933fe8-dfea-47b9-b610-dcfe642e1cc4 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:18:31.842714 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-71933fe8-dfea-47b9-b610-dcfe642e1cc4 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/f77bdcde-6735-49c1-82e1-4b2936088649 HTTP/1.1" status: 204 len: 168 time: 0.2085619 May 20 04:18:42.592059 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-f291104a-9455-465f-982d-24ad9873e6c0 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:18:44.956025 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:18:45.081049 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-753a2da1-e77f-414a-a62d-82399a8f4862 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:18:46.135062 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:18:47.776090 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:18:57.786826 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 5 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:19:12.596777 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-4c3c53ae-0e21-4406-addc-a79b5840a0c9 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:19:14.956966 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:19:15.074765 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-2343f967-3a4d-4c15-9f60-b45b6d8805d6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:19:16.143512 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:19:17.774734 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:19:34.791679 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 5 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:19:42.598633 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-140fc122-d024-48fd-96d9-7ef7e912eb73 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:19:44.959612 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:19:45.082463 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-3d33f60a-dd40-4dee-bbcf-3f7ed8e04a0b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:19:46.140689 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:19:47.779856 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:20:05.567707 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988'] for agent, after_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:20:08.884887 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 60322) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:20:09.011681 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-78707d81-8445-4851-b392-205eb4012df0 admin admin] 10.222.0.39 "GET /v2.0/agents HTTP/1.1" status: 200 len: 4303 time: 0.1256900 May 20 04:20:11.139795 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 60332) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:20:11.198334 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-46c858e9-1e73-4d16-8430-7e3e6ea1ec37 admin admin] 10.222.0.39 "GET /v2.0/agents HTTP/1.1" status: 200 len: 4303 time: 0.0573945 May 20 04:20:11.796381 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:20:12.596840 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-6f6e8444-b990-4423-b184-7246c6acadff None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:20:14.957495 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:20:15.076604 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-f84f985c-7979-4efb-b626-56bf1ebb921f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:20:16.145862 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:20:17.775422 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:20:35.531849 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:20:42.603602 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-7fae7f8a-8d17-45e8-947d-0e152085fd45 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:20:44.957593 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:20:45.072730 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-02a26f45-8cff-41aa-8b55-ceac1cd87fd1 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:20:46.146927 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:20:47.884300 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:20:48.801072 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:21:05.522385 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:12.586407 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-7c832c9a-70be-4146-9af9-633e71a809c7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:14.951848 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:15.077859 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-617f5883-6f20-451e-9a79-62d199165c36 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:16.148357 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:17.784636 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:25.800177 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:21:35.519297 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:37.360418 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 60430) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:37.562949 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Request body: {'network': {'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-network', 'tenant_id': '9fafdf3a29024ac588e5b24719735677'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:21:37.567569 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:37.572691 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Usage tracker for resource:network and tenant:9fafdf3a29024ac588e5b24719735677 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:37.576555 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:37.584249 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:37.595106 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Made reservation on behalf of 9fafdf3a29024ac588e5b24719735677 for: {'network': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:21:37.597999 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-585852'] for network, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:37.611029 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:37.629751 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:37.634455 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Usage tracker for resource:security_group_rule and tenant:9fafdf3a29024ac588e5b24719735677 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:37.637529 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:37.644569 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:37.730267 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:37.734649 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:37.737754 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:37.767336 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 847} {{(pid=91644) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 20 04:21:37.779849 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:37.779849 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.db.segments_db [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Added segment 9d1b8515-c35d-48e6-86ad-4eff412e924a of type vlan for network 64bcea8a-6d35-406b-a823-fa1185008174 May 20 04:21:37.806345 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8760688516720'] for network, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:37.810777 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=4d590d22-d6da-4a07-99f3-1768191cb1d6,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:37.819718 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:37.820080 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:37.820408 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:37.820833 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:37.821078 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:37.821403 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:37.821710 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:37.822035 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:37.822355 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:37.822675 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:37.823031 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:37.823386 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Tracked resource versions report: May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: pushed versions: May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: { 'AddressGroup': {'1.2'}, May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'ConntrackHelper': {'1.0'}, May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'Log': {'1.0'}, May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'Network': {'1.1'}, May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'Port': {'1.7'}, May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'PortForwarding': {'1.2'}, May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'QosPolicy': {'1.8'}, May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'SecurityGroup': {'1.4'}, May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'SecurityGroupRule': {'1.2'}, May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'SubPort': {'1.0'}, May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'Subnet': {'1.1'}, May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'Trunk': {'1.1'}} May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: consumer versions: May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: { DHCP agent@dv-cinder-782405-9-iscsi: {}, May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: HyperV agent@hv-cinder-78240: {}, May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: L3 agent@dv-cinder-782405-9-iscsi: {}, May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: Metadata agent@dv-cinder-782405-9-iscsi: {}, May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: Metering agent@dv-cinder-782405-9-iscsi: {}, May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: Open vSwitch agent@dv-cinder-782405-9-iscsi: { 'AddressGroup': '1.2', May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'ConntrackHelper': '1.0', May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'Log': '1.0', May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'Network': '1.1', May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'Port': '1.7', May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'PortForwarding': '1.2', May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'QosPolicy': '1.8', May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'SecurityGroup': '1.4', May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'SecurityGroupRule': '1.2', May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'SubPort': '1.0', May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'Subnet': '1.1', May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: 'Trunk': '1.1'}} May 20 04:21:37.824305 dv-cinder-782405-9-iscsi neutron-server[91644]: {{(pid=91644) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 20 04:21:37.839498 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.102s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:37.866524 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915'] for network, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:37.868022 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:21:37.868595 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:37.881176 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d80dcdb0-8fb9-4a39-87dc-1cc0d2c3f109 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:37.973720 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Pushing event updated for resources: {'Network': ['ID=64bcea8a-6d35-406b-a823-fa1185008174,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:37.980515 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] 10.222.0.39 "POST /v2.0/networks HTTP/1.1" status: 201 len: 890 time: 0.6190856 May 20 04:21:37.983550 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60478) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:37.987256 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Network 64bcea8a-6d35-406b-a823-fa1185008174 details requested by agent ovs-agent-dv-cinder-782405-9-iscsi with host dv-cinder-782405-9-iscsi {{(pid=91645) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 20 04:21:37.987872 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:37.995146 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Request body: {'subnet': {'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'cidr': '172.20.0.0/28', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-subnet', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'enable_dhcp': True, 'ip_version': 4}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:21:38.000614 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:38.004577 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Usage tracker for resource:subnet and tenant:9fafdf3a29024ac588e5b24719735677 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:38.007572 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:38.014549 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:38.025501 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Made reservation on behalf of 9fafdf3a29024ac588e5b24719735677 for: {'subnet': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:21:38.027962 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Notify callbacks [] for subnet, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:38.442031 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868'] for subnet, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:38.447695 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9c6d9180-1c3b-47b3-99b4-96156317facb admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:38.502003 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Pushing event updated for resources: {'Subnet': ['ID=203a3962-2282-4f51-ac92-c9695b5c0cfe,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:38.510102 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:38.510494 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:38.510841 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:38.511164 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:38.511503 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:38.511839 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:38.512118 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:38.512536 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:38.512800 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:38.513107 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:38.513447 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:38.513744 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Tracked resource versions report: May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: pushed versions: May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: { 'AddressGroup': {'1.2'}, May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'ConntrackHelper': {'1.0'}, May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'Log': {'1.0'}, May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'Network': {'1.1'}, May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'Port': {'1.7'}, May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'PortForwarding': {'1.2'}, May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'QosPolicy': {'1.8'}, May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'SecurityGroup': {'1.4'}, May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'SecurityGroupRule': {'1.2'}, May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'SubPort': {'1.0'}, May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'Subnet': {'1.1'}, May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'Trunk': {'1.1'}} May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: consumer versions: May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: { DHCP agent@dv-cinder-782405-9-iscsi: {}, May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: HyperV agent@hv-cinder-78240: {}, May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: L3 agent@dv-cinder-782405-9-iscsi: {}, May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: Metadata agent@dv-cinder-782405-9-iscsi: {}, May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: Metering agent@dv-cinder-782405-9-iscsi: {}, May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: Open vSwitch agent@dv-cinder-782405-9-iscsi: { 'AddressGroup': '1.2', May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'ConntrackHelper': '1.0', May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'Log': '1.0', May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'Network': '1.1', May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'Port': '1.7', May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'PortForwarding': '1.2', May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'QosPolicy': '1.8', May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'SecurityGroup': '1.4', May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'SecurityGroupRule': '1.2', May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'SubPort': '1.0', May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'Subnet': '1.1', May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: 'Trunk': '1.1'}} May 20 04:21:38.514839 dv-cinder-782405-9-iscsi neutron-server[91643]: {{(pid=91643) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 20 04:21:38.522958 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.075s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:38.545963 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Checking segment: {'id': '9d1b8515-c35d-48e6-86ad-4eff412e924a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 847, 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91643) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:21:38.546319 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Network hostable DHCP agents. Network: 64bcea8a-6d35-406b-a823-fa1185008174, hostable agents: ['fffae25a-f450-4182-8f9e-ac29d143f6c9'], hosted agents: [] {{(pid=91643) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 20 04:21:38.564817 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Network 64bcea8a-6d35-406b-a823-fa1185008174 is scheduled to be hosted by DHCP agent fffae25a-f450-4182-8f9e-ac29d143f6c9 with binding_index 1 {{(pid=91643) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 20 04:21:38.579599 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-782405-9-iscsi, 64bcea8a-6d35-406b-a823-fa1185008174) {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:21:38.584190 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:21:38.584944 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:38.596047 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-0ff785fc-bb3e-4b8b-b857-aa1186f5a52b None None] Network 64bcea8a-6d35-406b-a823-fa1185008174 requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:21:38.613067 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 60516) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:38.619585 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Request body: {'router': {'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:21:38.624891 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:38.625388 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-64529bb0-6a7c-4428-ad6c-d6353d445aca admin admin] 10.222.0.39 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 853 time: 0.6347859 May 20 04:21:38.630799 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Usage tracker for resource:router and tenant:9fafdf3a29024ac588e5b24719735677 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:38.634216 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:38.640806 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:38.657255 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-8071540', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-260712'] for router, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:38.661168 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-8056436', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-8053301', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-7881139', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-260707'] for router, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:38.679493 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:38.910735 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcp85c0b5c8-c799-5455-967d-3e9c2c8b4839-64bcea8a-6d35-406b-a823-fa1185008174', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe'}]}} from dv-cinder-782405-9-iscsi. {{(pid=91645) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 20 04:21:38.911699 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:38.954316 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60536) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:38.979539 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-8053308', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-8071430', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-260685'] for router, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:39.181134 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.db_base_plugin_common [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Allocated IP 172.20.0.2 (64bcea8a-6d35-406b-a823-fa1185008174/203a3962-2282-4f51-ac92-c9695b5c0cfe/ca916c39-7b0c-4a56-94cc-6d171114540b) {{(pid=91645) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:21:39.288944 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60552) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:39.344577 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Request body: {'network': {'name': 'tempest-AttachVolumeNegativeTest-1929799074-network', 'tenant_id': '51fcbc78284541db832e4780e76158af'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:21:39.351610 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:39.355885 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:39.359591 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Usage tracker for resource:network and tenant:51fcbc78284541db832e4780e76158af is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:39.363158 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:39.369807 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:39.384648 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Made reservation on behalf of 51fcbc78284541db832e4780e76158af for: {'network': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:21:39.388461 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-585852'] for network, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:39.392594 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:39.403905 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:39.421446 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:39.426172 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Usage tracker for resource:security_group_rule and tenant:51fcbc78284541db832e4780e76158af is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:39.435102 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:39.445489 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:39.493735 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Notify callbacks [] for port, precommit_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:39.507290 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Transition to ACTIVE for port object ca916c39-7b0c-4a56-94cc-6d171114540b will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:21:39.535910 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Transition to ACTIVE for port object ca916c39-7b0c-4a56-94cc-6d171114540b will not be triggered until provisioned by entity DHCP. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:21:39.543602 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:39.549047 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:39.582842 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:39.589478 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:39.597932 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-105194a5-3cbd-473f-8585-8bd325d6ea6e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:39.606345 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Network 64bcea8a-6d35-406b-a823-fa1185008174 is already hosted by enough agents. {{(pid=91645) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:21:39.614511 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.base_scheduler [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Resources bound (agent type, host, resource id): {{(pid=91645) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:21:39.615817 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Allocated IP 172.27.0.202 (712ce8c4-8feb-4719-bf13-e06992125cd9/e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5/063b884b-9614-48ee-9ca9-0c68b3e5f35f) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:21:39.624795 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Received port after_create {{(pid=91645) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:21:39.663654 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:39.672624 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.075s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:39.682772 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 836} {{(pid=91643) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 20 04:21:39.708903 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:39.709236 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.segments_db [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Added segment 6fe99325-a72c-48bf-abc7-9bbcc4b753ff of type vlan for network 54fc8d38-193b-49c4-b873-c5353723791f May 20 04:21:39.744983 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Attempting to bind port ca916c39-7b0c-4a56-94cc-6d171114540b on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:21:39.745638 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Attempting to bind port ca916c39-7b0c-4a56-94cc-6d171114540b by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '9d1b8515-c35d-48e6-86ad-4eff412e924a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 847, 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:21:39.746334 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Attempting to bind port ca916c39-7b0c-4a56-94cc-6d171114540b on network 64bcea8a-6d35-406b-a823-fa1185008174 {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:21:39.756098 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8760688516720'] for network, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:39.756832 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 21, 15), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:21:39.757409 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Checking segment: {'id': '9d1b8515-c35d-48e6-86ad-4eff412e924a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 847, 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:21:39.757810 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:39.758205 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Bound using segment: {'id': '9d1b8515-c35d-48e6-86ad-4eff412e924a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 847, 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:21:39.758754 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Bound port: ca916c39-7b0c-4a56-94cc-6d171114540b, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '9d1b8515-c35d-48e6-86ad-4eff412e924a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 847, 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:21:39.759508 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:39.822925 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915'] for network, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:39.824660 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:21:39.825510 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:39.830561 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:39.845108 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60574) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:39.868321 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:39.879102 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Request body: {'network': {'name': 'tempest-DeleteServersTestJSON-80902257-network', 'tenant_id': '0b29d684ade544028fa17828cde46d56'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:21:39.887883 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:39.892774 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Usage tracker for resource:network and tenant:0b29d684ade544028fa17828cde46d56 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:39.896040 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] For port ca916c39-7b0c-4a56-94cc-6d171114540b, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:21:39.896649 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:39.902862 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] For port ca916c39-7b0c-4a56-94cc-6d171114540b, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=ca916c39-7b0c-4a56-94cc-6d171114540b,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:21:39.904083 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:39.918547 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:39.919342 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Pushing event updated for resources: {'Port': ['ID=ca916c39-7b0c-4a56-94cc-6d171114540b,revision_number=1']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:39.931657 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:39.936690 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:39.940848 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:39.941251 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:39.941601 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:39.941898 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:39.942223 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:39.942541 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:39.942861 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:39.943245 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:39.943530 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:39.943841 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:39.944154 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Tracked resource versions report: May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: pushed versions: May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: { 'AddressGroup': {'1.2'}, May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'ConntrackHelper': {'1.0'}, May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'Log': {'1.0'}, May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'Network': {'1.1'}, May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'Port': {'1.7'}, May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'PortForwarding': {'1.2'}, May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'QosPolicy': {'1.8'}, May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'SecurityGroup': {'1.4'}, May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'SecurityGroupRule': {'1.2'}, May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'SubPort': {'1.0'}, May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'Subnet': {'1.1'}, May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'Trunk': {'1.1'}} May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: consumer versions: May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: { DHCP agent@dv-cinder-782405-9-iscsi: {}, May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: HyperV agent@hv-cinder-78240: {}, May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: L3 agent@dv-cinder-782405-9-iscsi: {}, May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: Metadata agent@dv-cinder-782405-9-iscsi: {}, May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: Metering agent@dv-cinder-782405-9-iscsi: {}, May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: Open vSwitch agent@dv-cinder-782405-9-iscsi: { 'AddressGroup': '1.2', May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'ConntrackHelper': '1.0', May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'Log': '1.0', May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'Network': '1.1', May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'Port': '1.7', May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'PortForwarding': '1.2', May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'QosPolicy': '1.8', May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'SecurityGroup': '1.4', May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'SecurityGroupRule': '1.2', May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'SubPort': '1.0', May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'Subnet': '1.1', May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: 'Trunk': '1.1'}} May 20 04:21:39.945227 dv-cinder-782405-9-iscsi neutron-server[91645]: {{(pid=91645) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 20 04:21:39.949559 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:39.971729 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.415s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:39.981669 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:40.002819 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] 10.222.0.39 "POST /v2.0/networks HTTP/1.1" status: 201 len: 888 time: 1.0468543 May 20 04:21:40.004249 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Pushing event updated for resources: {'Network': ['ID=54fc8d38-193b-49c4-b873-c5353723791f,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:40.007025 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60584) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:40.008469 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Made reservation on behalf of 0b29d684ade544028fa17828cde46d56 for: {'network': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:21:40.033743 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-585852'] for network, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:40.045852 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.215s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:40.049854 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-5c6392b8-6388-4fc8-af6b-a7f6fa7356c6 admin admin] Request body: {'subnet': {'network_id': '54fc8d38-193b-49c4-b873-c5353723791f', 'cidr': '172.20.0.0/28', 'name': 'tempest-AttachVolumeNegativeTest-1929799074-subnet', 'tenant_id': '51fcbc78284541db832e4780e76158af', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:21:40.064635 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:40.068983 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:21:40.082399 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Request body: {'network': {'name': 'tempest-ServerRescueNegativeTestJSON-1026520542-network', 'tenant_id': 'bec56d8703cf47f3b7193b7d1f55396d'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:21:40.083331 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:21:40.086870 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-5c6392b8-6388-4fc8-af6b-a7f6fa7356c6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:40.087363 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:21:40.091344 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Network 54fc8d38-193b-49c4-b873-c5353723791f details requested by agent ovs-agent-dv-cinder-782405-9-iscsi with host dv-cinder-782405-9-iscsi {{(pid=91645) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 20 04:21:40.093251 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:40.097318 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Usage tracker for resource:security_group_rule and tenant:0b29d684ade544028fa17828cde46d56 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:40.100680 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:40.106912 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:40.203555 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:40.207951 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:40.212821 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-5c6392b8-6388-4fc8-af6b-a7f6fa7356c6 admin admin] Usage tracker for resource:subnet and tenant:51fcbc78284541db832e4780e76158af is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:40.215163 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:40.216099 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-5c6392b8-6388-4fc8-af6b-a7f6fa7356c6 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:40.235368 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 814} {{(pid=91643) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 20 04:21:40.245534 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:40.245869 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.segments_db [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Added segment 3c4a2238-8a32-4d92-a1b1-8af91c1c27a7 of type vlan for network a009b619-8ab3-40e3-b5f3-9cc97ccd3415 May 20 04:21:40.274349 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8760688516720'] for network, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:40.313351 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-5c6392b8-6388-4fc8-af6b-a7f6fa7356c6 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:40.315750 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Usage tracker for resource:network and tenant:bec56d8703cf47f3b7193b7d1f55396d is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:40.319286 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:40.326076 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:40.330783 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915'] for network, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:40.357688 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=06880d35-d748-4b5a-88d1-1192a88678c7,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:40.376779 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:40.379409 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:21:40.380344 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:40.403324 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:40.406276 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-2b61821f-6984-41c7-a92e-f007a48cf938 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:40.510993 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Pushing event updated for resources: {'Network': ['ID=a009b619-8ab3-40e3-b5f3-9cc97ccd3415,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:40.521051 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] 10.222.0.39 "POST /v2.0/networks HTTP/1.1" status: 201 len: 883 time: 1.2305000 May 20 04:21:40.522537 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.116s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:40.524955 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-5c6392b8-6388-4fc8-af6b-a7f6fa7356c6 admin admin] Made reservation on behalf of 51fcbc78284541db832e4780e76158af for: {'subnet': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:21:40.527525 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5c6392b8-6388-4fc8-af6b-a7f6fa7356c6 admin admin] Notify callbacks [] for subnet, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:40.618419 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Network a009b619-8ab3-40e3-b5f3-9cc97ccd3415 details requested by agent ovs-agent-dv-cinder-782405-9-iscsi with host dv-cinder-782405-9-iscsi {{(pid=91645) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 20 04:21:40.660267 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 60612) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:40.694081 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Pushing event updated for resources: {'Port': ['ID=063b884b-9614-48ee-9ca9-0c68b3e5f35f,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:40.844736 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Made reservation on behalf of bec56d8703cf47f3b7193b7d1f55396d for: {'network': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:21:40.859339 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-585852'] for network, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:40.884542 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:40.927236 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-6c432f4c-92f8-453b-968a-5b23526db9e2 admin admin] Request body: {'subnet': {'network_id': 'a009b619-8ab3-40e3-b5f3-9cc97ccd3415', 'cidr': '172.20.0.0/28', 'name': 'tempest-DeleteServersTestJSON-80902257-subnet', 'tenant_id': '0b29d684ade544028fa17828cde46d56', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:21:40.937216 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-6c432f4c-92f8-453b-968a-5b23526db9e2 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:40.942038 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:40.950140 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Usage tracker for resource:security_group_rule and tenant:bec56d8703cf47f3b7193b7d1f55396d is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:40.951419 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-6c432f4c-92f8-453b-968a-5b23526db9e2 admin admin] Usage tracker for resource:subnet and tenant:0b29d684ade544028fa17828cde46d56 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:40.955161 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-6c432f4c-92f8-453b-968a-5b23526db9e2 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:40.962405 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:40.965660 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-6c432f4c-92f8-453b-968a-5b23526db9e2 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:40.978765 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Pushing event updated for resources: {'Port': ['ID=ca916c39-7b0c-4a56-94cc-6d171114540b,revision_number=2']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:40.979306 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 1.029s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:40.981071 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:40.985216 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 1.007s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:41.077927 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:41.457691 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:41.475115 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:41.484594 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:41.529511 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5c6392b8-6388-4fc8-af6b-a7f6fa7356c6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868'] for subnet, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:41.533456 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 813} {{(pid=91643) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 20 04:21:41.544951 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=54593a4a-1755-4771-9f21-b88328299a32,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:41.553080 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-6c432f4c-92f8-453b-968a-5b23526db9e2 admin admin] Made reservation on behalf of 0b29d684ade544028fa17828cde46d56 for: {'subnet': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:21:41.560181 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6c432f4c-92f8-453b-968a-5b23526db9e2 admin admin] Notify callbacks [] for subnet, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:41.564017 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.079s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:41.572422 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-c07f55d5-5609-454b-971f-f74a83d9d176 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.028s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:41.613869 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:41.614623 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.segments_db [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Added segment 0032d9a3-07bf-489f-bb9d-03ab3ff14f98 of type vlan for network 36b37213-8439-447d-9eba-a78baf7c698e May 20 04:21:41.663451 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-5c6392b8-6388-4fc8-af6b-a7f6fa7356c6 admin admin] Checking segment: {'id': '6fe99325-a72c-48bf-abc7-9bbcc4b753ff', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 836, 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91643) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:21:41.668621 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-5c6392b8-6388-4fc8-af6b-a7f6fa7356c6 admin admin] Network hostable DHCP agents. Network: 54fc8d38-193b-49c4-b873-c5353723791f, hostable agents: ['fffae25a-f450-4182-8f9e-ac29d143f6c9'], hosted agents: [] {{(pid=91643) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 20 04:21:41.696577 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5c6392b8-6388-4fc8-af6b-a7f6fa7356c6 admin admin] Pushing event updated for resources: {'Subnet': ['ID=0f918085-8d33-4e36-a59e-7f270a2b1874,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:41.721949 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-5c6392b8-6388-4fc8-af6b-a7f6fa7356c6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.149s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:41.723917 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8760688516720'] for network, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:41.755317 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-5c6392b8-6388-4fc8-af6b-a7f6fa7356c6 admin admin] Network 54fc8d38-193b-49c4-b873-c5353723791f is scheduled to be hosted by DHCP agent fffae25a-f450-4182-8f9e-ac29d143f6c9 with binding_index 1 {{(pid=91643) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 20 04:21:41.772902 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-5c6392b8-6388-4fc8-af6b-a7f6fa7356c6 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-782405-9-iscsi, 54fc8d38-193b-49c4-b873-c5353723791f) {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:21:41.775562 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915'] for network, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:41.778430 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:21:41.779116 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:41.782214 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-2a187b0b-c109-4ad7-b1e4-33ae0751714b None None] Network 54fc8d38-193b-49c4-b873-c5353723791f requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:21:41.784187 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-5c6392b8-6388-4fc8-af6b-a7f6fa7356c6 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:21:41.784761 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5c6392b8-6388-4fc8-af6b-a7f6fa7356c6 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:41.787875 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:41.952787 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 60642) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:42.013613 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-8071551', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-8053391'] for router, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:42.057719 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-218ff57e-9acb-41ce-bd4f-20868a50392c None None] Network 54fc8d38-193b-49c4-b873-c5353723791f requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:21:42.116422 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Pushing event updated for resources: {'Network': ['ID=36b37213-8439-447d-9eba-a78baf7c698e,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:42.127258 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.338s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:42.171905 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] 10.222.0.39 "POST /v2.0/networks HTTP/1.1" status: 201 len: 892 time: 2.3206005 May 20 04:21:42.172975 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60658) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:42.176667 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-5c6392b8-6388-4fc8-af6b-a7f6fa7356c6 admin admin] 10.222.0.39 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 852 time: 2.1323254 May 20 04:21:42.190944 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-83eea8cc-0fe2-4097-b6ac-04d4078f9c56 admin admin] Request body: {'subnet': {'network_id': '36b37213-8439-447d-9eba-a78baf7c698e', 'cidr': '172.20.0.0/28', 'name': 'tempest-ServerRescueNegativeTestJSON-1026520542-subnet', 'tenant_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:21:42.200891 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-83eea8cc-0fe2-4097-b6ac-04d4078f9c56 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:42.209946 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-83eea8cc-0fe2-4097-b6ac-04d4078f9c56 admin admin] Usage tracker for resource:subnet and tenant:bec56d8703cf47f3b7193b7d1f55396d is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:42.214441 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Network 36b37213-8439-447d-9eba-a78baf7c698e details requested by agent ovs-agent-dv-cinder-782405-9-iscsi with host dv-cinder-782405-9-iscsi {{(pid=91645) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 20 04:21:42.217999 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-83eea8cc-0fe2-4097-b6ac-04d4078f9c56 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:42.230446 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-83eea8cc-0fe2-4097-b6ac-04d4078f9c56 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:42.252615 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-83eea8cc-0fe2-4097-b6ac-04d4078f9c56 admin admin] Made reservation on behalf of bec56d8703cf47f3b7193b7d1f55396d for: {'subnet': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:21:42.256311 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-83eea8cc-0fe2-4097-b6ac-04d4078f9c56 admin admin] Notify callbacks [] for subnet, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:42.307456 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-fcfa7b9f-3c2a-4aa6-ac1d-869ec8bdd4b4 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:42.485125 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-fcfa7b9f-3c2a-4aa6-ac1d-869ec8bdd4b4 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:42.536352 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-fcfa7b9f-3c2a-4aa6-ac1d-869ec8bdd4b4 admin admin] Usage tracker for resource:security_group_rule and tenant:a63b4ae3a2eb4a13a9fd5412cc97b98e is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:42.551581 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-fcfa7b9f-3c2a-4aa6-ac1d-869ec8bdd4b4 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:42.573939 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-fcfa7b9f-3c2a-4aa6-ac1d-869ec8bdd4b4 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:42.643004 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-ee94bbf7-7721-41d3-9e53-b93cc807a41c None None] Network 64bcea8a-6d35-406b-a823-fa1185008174 requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:21:42.851672 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-a4679cb2-741e-4c08-a35a-0a4f7ac01649 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:42.864034 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6c432f4c-92f8-453b-968a-5b23526db9e2 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868'] for subnet, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:42.879944 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-24cc019d-ee04-45b4-90db-d4d855572891 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:42.928869 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-83eea8cc-0fe2-4097-b6ac-04d4078f9c56 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868'] for subnet, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:42.930016 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-0cbecb7e-dc7a-41b1-8ef4-2abd89d9fab9 None None] Provisioning for port ca916c39-7b0c-4a56-94cc-6d171114540b completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:21:42.934218 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60704) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:42.935281 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-5c6392b8-6388-4fc8-af6b-a7f6fa7356c6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:42.968396 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc is scheduled to L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 with binding_index 1 {{(pid=91644) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 20 04:21:42.975977 dv-cinder-782405-9-iscsi neutron-server[91643]: WARNING keystonemiddleware.auth_token [-] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. May 20 04:21:43.080565 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-6c432f4c-92f8-453b-968a-5b23526db9e2 admin admin] Checking segment: {'id': '3c4a2238-8a32-4d92-a1b1-8af91c1c27a7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 814, 'network_id': 'a009b619-8ab3-40e3-b5f3-9cc97ccd3415'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:21:43.080947 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-6c432f4c-92f8-453b-968a-5b23526db9e2 admin admin] Network hostable DHCP agents. Network: a009b619-8ab3-40e3-b5f3-9cc97ccd3415, hostable agents: ['fffae25a-f450-4182-8f9e-ac29d143f6c9'], hosted agents: [] {{(pid=91644) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 20 04:21:43.083737 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6c432f4c-92f8-453b-968a-5b23526db9e2 admin admin] Pushing event updated for resources: {'Subnet': ['ID=b9f2dbbb-b724-44b9-878b-2267c4d95aee,revision_number=0']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:43.097118 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-6c432f4c-92f8-453b-968a-5b23526db9e2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.217s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:43.132887 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-fcfa7b9f-3c2a-4aa6-ac1d-869ec8bdd4b4 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:43.144432 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-fcfa7b9f-3c2a-4aa6-ac1d-869ec8bdd4b4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:43.166026 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-83eea8cc-0fe2-4097-b6ac-04d4078f9c56 admin admin] Pushing event updated for resources: {'Subnet': ['ID=b25ae9c8-e11d-45ee-bde2-332cc1538602,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:43.178391 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-83eea8cc-0fe2-4097-b6ac-04d4078f9c56 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.243s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:43.208435 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-83eea8cc-0fe2-4097-b6ac-04d4078f9c56 admin admin] Checking segment: {'id': '0032d9a3-07bf-489f-bb9d-03ab3ff14f98', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 813, 'network_id': '36b37213-8439-447d-9eba-a78baf7c698e'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91643) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:21:43.208816 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-83eea8cc-0fe2-4097-b6ac-04d4078f9c56 admin admin] Network hostable DHCP agents. Network: 36b37213-8439-447d-9eba-a78baf7c698e, hostable agents: ['fffae25a-f450-4182-8f9e-ac29d143f6c9'], hosted agents: [] {{(pid=91643) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 20 04:21:43.221903 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device ca916c39-7b0c-4a56-94cc-6d171114540b up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:21:43.259656 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:43.282658 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-6c432f4c-92f8-453b-968a-5b23526db9e2 admin admin] Network a009b619-8ab3-40e3-b5f3-9cc97ccd3415 is scheduled to be hosted by DHCP agent fffae25a-f450-4182-8f9e-ac29d143f6c9 with binding_index 1 {{(pid=91644) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 20 04:21:43.302568 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-83eea8cc-0fe2-4097-b6ac-04d4078f9c56 admin admin] Network 36b37213-8439-447d-9eba-a78baf7c698e is scheduled to be hosted by DHCP agent fffae25a-f450-4182-8f9e-ac29d143f6c9 with binding_index 1 {{(pid=91643) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 20 04:21:43.320484 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-6c432f4c-92f8-453b-968a-5b23526db9e2 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-782405-9-iscsi, a009b619-8ab3-40e3-b5f3-9cc97ccd3415) {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:21:43.338799 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-6c432f4c-92f8-453b-968a-5b23526db9e2 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:21:43.339422 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6c432f4c-92f8-453b-968a-5b23526db9e2 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:43.345552 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-11918be3-7293-4be6-b4ee-4e48cd814799 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:21:43.349070 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-fcfa7b9f-3c2a-4aa6-ac1d-869ec8bdd4b4 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=a63b4ae3a2eb4a13a9fd5412cc97b98e&name=default HTTP/1.1" status: 200 len: 2975 time: 1.3928158 May 20 04:21:43.349748 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:21:43.351344 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fcfa7b9f-3c2a-4aa6-ac1d-869ec8bdd4b4 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=203ad64c-e14c-4b87-b7ef-d5240d01e3c3,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:43.352324 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-11918be3-7293-4be6-b4ee-4e48cd814799 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Synchronizing usage tracker for tenant:51fcbc78284541db832e4780e76158af on resource:network {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:21:43.356161 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-83eea8cc-0fe2-4097-b6ac-04d4078f9c56 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-782405-9-iscsi, 36b37213-8439-447d-9eba-a78baf7c698e) {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:21:43.359028 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Sync routers for ids ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'] in dv-cinder-782405-9-iscsi {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:21:43.361279 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60714) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:43.372695 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Routers {'5a3f69b9-fd58-499f-bec9-75215cd7f8fc'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:21:43.373340 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:21:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:21:43.386293 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-1b8b4e91-bd78-47cd-9711-0b8841ddabce None None] Network a009b619-8ab3-40e3-b5f3-9cc97ccd3415 requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:21:43.398638 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-83eea8cc-0fe2-4097-b6ac-04d4078f9c56 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:21:43.399102 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-83eea8cc-0fe2-4097-b6ac-04d4078f9c56 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:43.414006 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-4cf29e34-60dd-4ed6-bdd4-37badbfb5d3f None None] Network 36b37213-8439-447d-9eba-a78baf7c698e requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:21:43.464624 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-83eea8cc-0fe2-4097-b6ac-04d4078f9c56 admin admin] 10.222.0.39 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 856 time: 1.2854133 May 20 04:21:43.469357 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60724) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:43.478263 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-fcfa7b9f-3c2a-4aa6-ac1d-869ec8bdd4b4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.219s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:43.494938 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-6c432f4c-92f8-453b-968a-5b23526db9e2 admin admin] 10.222.0.39 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 847 time: 2.8021019 May 20 04:21:43.497459 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Request body: {'router': {'name': 'tempest-ServerRescueNegativeTestJSON-1026520542-router', 'tenant_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:21:43.497627 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Notify callbacks [] for router, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:43.500691 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] 10.222.0.39 "POST /v2.0/routers HTTP/1.1" status: 201 len: 929 time: 4.8854461 May 20 04:21:43.501039 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 60728) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:43.518509 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6553cfb0-7b48-43db-8efe-6b5b9c3132b3 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:43.520116 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-11918be3-7293-4be6-b4ee-4e48cd814799 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Unset dirty status for tenant:51fcbc78284541db832e4780e76158af on resource:network {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:21:43.521248 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-11918be3-7293-4be6-b4ee-4e48cd814799 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/networks?id=54fc8d38-193b-49c4-b873-c5353723791f HTTP/1.1" status: 200 len: 815 time: 0.5552552 May 20 04:21:43.522146 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:43.533420 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Usage tracker for resource:router and tenant:bec56d8703cf47f3b7193b7d1f55396d is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:43.540980 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:43.594153 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-1b7cf130-8a12-4a29-a24b-8a970b3801c1 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/quotas/51fcbc78284541db832e4780e76158af HTTP/1.1" status: 200 len: 375 time: 0.0545430 May 20 04:21:43.595983 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:43.733645 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6553cfb0-7b48-43db-8efe-6b5b9c3132b3 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:43.749915 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port ca916c39-7b0c-4a56-94cc-6d171114540b completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:21:43.756701 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-75221afd-269a-4f86-bbbd-a86ea3b70e0c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=51fcbc78284541db832e4780e76158af&fields=id HTTP/1.1" status: 200 len: 210 time: 0.1483557 May 20 04:21:43.771529 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Set router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:21:43.829471 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port ca916c39-7b0c-4a56-94cc-6d171114540b triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:21:43.839756 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:43.953582 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-8071540', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-260712'] for router, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:43.957632 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-8056436', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-8053301', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-7881139', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-260707'] for router, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:43.964838 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-d5fbc77d-2e87-407f-86e6-25e09616cf32 None None] Provisioning complete for port ca916c39-7b0c-4a56-94cc-6d171114540b triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:21:43.965413 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-d5fbc77d-2e87-407f-86e6-25e09616cf32 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:43.986925 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6553cfb0-7b48-43db-8efe-6b5b9c3132b3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:44.021284 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:44.069947 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:44.143897 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 60750) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:44.174568 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6553cfb0-7b48-43db-8efe-6b5b9c3132b3 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=203ad64c-e14c-4b87-b7ef-d5240d01e3c3,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:44.177371 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-6553cfb0-7b48-43db-8efe-6b5b9c3132b3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.156s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:44.180448 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6553cfb0-7b48-43db-8efe-6b5b9c3132b3 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:44.181748 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-6553cfb0-7b48-43db-8efe-6b5b9c3132b3 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/203ad64c-e14c-4b87-b7ef-d5240d01e3c3 HTTP/1.1" status: 204 len: 168 time: 0.8182571 May 20 04:21:44.260622 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-72879d34-01a6-48c0-be5e-7d3502b3d1c2 service neutron] http://10.222.0.39:9696/v2.0/extensions returned with HTTP 200 {{(pid=91643) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 20 04:21:44.269532 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-72879d34-01a6-48c0-be5e-7d3502b3d1c2 service neutron] 10.222.0.39 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.1035891 May 20 04:21:44.348888 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:21:40Z', 'revision_number': 3, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port': {'id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:e4:9a:7c', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:39Z', 'updated_at': '2021-05-20T04:21:39Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:21:44.360030 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Checking router: 5a3f69b9-fd58-499f-bec9-75215cd7f8fc for host: dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:21:44.360702 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Updating router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc port 063b884b-9614-48ee-9ca9-0c68b3e5f35f binding host dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 20 04:21:44.452110 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c93632de-3407-4063-872d-3059b2ac2e2f admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:44.472856 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:44.520591 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-c93632de-3407-4063-872d-3059b2ac2e2f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:44.536594 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port ca916c39-7b0c-4a56-94cc-6d171114540b, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=ca916c39-7b0c-4a56-94cc-6d171114540b,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:21:44.544611 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:44.546264 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-c93632de-3407-4063-872d-3059b2ac2e2f admin admin] Usage tracker for resource:security_group_rule and tenant:a1d94e15f8bd4d5b8d1adc8835ff314d is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:44.557490 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-c93632de-3407-4063-872d-3059b2ac2e2f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:44.714214 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-c93632de-3407-4063-872d-3059b2ac2e2f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:44.742607 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-5447a417-e61b-4a03-b1f2-e33ed46023f3 None None] Network 36b37213-8439-447d-9eba-a78baf7c698e requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:21:44.793218 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:44.848564 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:44.867249 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-8053308', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-8071430', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-260685'] for router, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:44.877740 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60772) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:44.903906 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-afc5000c-f4b7-435a-ad82-30efce6ea089 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:44.957051 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=ca916c39-7b0c-4a56-94cc-6d171114540b,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:44.967960 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-afc5000c-f4b7-435a-ad82-30efce6ea089 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:44.971369 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-aaac6cc4-6747-4427-bc91-5ba5d44013fb None None] Network a009b619-8ab3-40e3-b5f3-9cc97ccd3415 requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:21:44.979813 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-afc5000c-f4b7-435a-ad82-30efce6ea089 admin admin] Usage tracker for resource:security_group_rule and tenant:6bbcfebed7b44b7d8d14d6a132471ff9 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:44.990447 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-878d8d47-0522-4afb-a7e6-8a50a0e46b0e service neutron] 10.222.0.39 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.7198784 May 20 04:21:44.992387 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-afc5000c-f4b7-435a-ad82-30efce6ea089 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:45.028552 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-afc5000c-f4b7-435a-ad82-30efce6ea089 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:45.121957 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.274s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:45.236775 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device ca916c39-7b0c-4a56-94cc-6d171114540b up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:21:45.266033 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c93632de-3407-4063-872d-3059b2ac2e2f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:45.270984 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Notify callbacks [] for port, precommit_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:45.289469 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.provisioning_blocks [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Transition to ACTIVE for port object 063b884b-9614-48ee-9ca9-0c68b3e5f35f will not be triggered until provisioned by entity L2. {{(pid=91646) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:21:45.299788 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:45.308029 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:45.316278 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c93632de-3407-4063-872d-3059b2ac2e2f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:45.329704 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-afc5000c-f4b7-435a-ad82-30efce6ea089 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:45.333835 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:45.356277 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:45.397080 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60792) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:45.398870 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-afc5000c-f4b7-435a-ad82-30efce6ea089 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:45.425353 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Attempting to bind port 063b884b-9614-48ee-9ca9-0c68b3e5f35f on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:21:45.425955 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Attempting to bind port 063b884b-9614-48ee-9ca9-0c68b3e5f35f by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}] {{(pid=91646) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:21:45.426078 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Attempting to bind port 063b884b-9614-48ee-9ca9-0c68b3e5f35f on network 712ce8c4-8feb-4719-bf13-e06992125cd9 {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:21:45.427245 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-bec8ac77-5f74-41d1-a7a9-4d772a159760 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:45.431795 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c93632de-3407-4063-872d-3059b2ac2e2f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d383c378-6b96-4b11-881b-1694db0caa3a,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:45.440059 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 21, 45), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:21:45.440433 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Checking segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91646) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:21:45.440783 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:45.440958 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:45.441353 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Bound using segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:21:45.441828 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Bound port: 063b884b-9614-48ee-9ca9-0c68b3e5f35f, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}}] {{(pid=91646) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:21:45.442201 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:45.460656 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-c93632de-3407-4063-872d-3059b2ac2e2f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:45.492655 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-607f95c8-48de-4b82-8554-fd706f67790b None None] Pushing event updated for resources: {'Port': ['ID=063b884b-9614-48ee-9ca9-0c68b3e5f35f,revision_number=2']} {{(pid=91646) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:45.532746 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-607f95c8-48de-4b82-8554-fd706f67790b None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:45.533080 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-607f95c8-48de-4b82-8554-fd706f67790b None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:45.533409 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-607f95c8-48de-4b82-8554-fd706f67790b None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:45.533731 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-607f95c8-48de-4b82-8554-fd706f67790b None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:45.534047 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-607f95c8-48de-4b82-8554-fd706f67790b None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:45.534426 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-607f95c8-48de-4b82-8554-fd706f67790b None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:45.534746 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-607f95c8-48de-4b82-8554-fd706f67790b None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:45.535061 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-607f95c8-48de-4b82-8554-fd706f67790b None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:45.535432 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-607f95c8-48de-4b82-8554-fd706f67790b None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:45.535752 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-607f95c8-48de-4b82-8554-fd706f67790b None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:45.536064 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-607f95c8-48de-4b82-8554-fd706f67790b None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:45.536373 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-607f95c8-48de-4b82-8554-fd706f67790b None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-607f95c8-48de-4b82-8554-fd706f67790b None None] Tracked resource versions report: May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: pushed versions: May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: { 'AddressGroup': {'1.2'}, May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'ConntrackHelper': {'1.0'}, May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'Log': {'1.0'}, May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'Network': {'1.1'}, May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'Port': {'1.7'}, May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'PortForwarding': {'1.2'}, May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'QosPolicy': {'1.8'}, May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'SecurityGroup': {'1.4'}, May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'SecurityGroupRule': {'1.2'}, May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'SubPort': {'1.0'}, May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'Subnet': {'1.1'}, May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'Trunk': {'1.1'}} May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: consumer versions: May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: { DHCP agent@dv-cinder-782405-9-iscsi: {}, May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: HyperV agent@hv-cinder-78240: {}, May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: L3 agent@dv-cinder-782405-9-iscsi: {}, May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: Metadata agent@dv-cinder-782405-9-iscsi: {}, May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: Metering agent@dv-cinder-782405-9-iscsi: {}, May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: Open vSwitch agent@dv-cinder-782405-9-iscsi: { 'AddressGroup': '1.2', May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'ConntrackHelper': '1.0', May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'Log': '1.0', May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'Network': '1.1', May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'Port': '1.7', May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'PortForwarding': '1.2', May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'QosPolicy': '1.8', May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'SecurityGroup': '1.4', May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'SecurityGroupRule': '1.2', May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'SubPort': '1.0', May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'Subnet': '1.1', May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: 'Trunk': '1.1'}} May 20 04:21:45.537434 dv-cinder-782405-9-iscsi neutron-server[91646]: {{(pid=91646) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 20 04:21:45.554476 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port ca916c39-7b0c-4a56-94cc-6d171114540b triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:21:45.554724 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-607f95c8-48de-4b82-8554-fd706f67790b None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.247s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:45.554976 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:45.582791 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:45.593284 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.db [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] For port 063b884b-9614-48ee-9ca9-0c68b3e5f35f, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91646) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:21:45.593900 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-afc5000c-f4b7-435a-ad82-30efce6ea089 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=6bbcfebed7b44b7d8d14d6a132471ff9&name=default HTTP/1.1" status: 200 len: 2975 time: 0.7156966 May 20 04:21:45.597009 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.db [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] For port 063b884b-9614-48ee-9ca9-0c68b3e5f35f, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=063b884b-9614-48ee-9ca9-0c68b3e5f35f,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91646) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:21:45.603313 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:45.607389 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 60798) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:45.608973 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-6d0351fa-a266-4e28-b4ff-583cbf6bb130 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:45.687558 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-607f95c8-48de-4b82-8554-fd706f67790b None None] Pushing event updated for resources: {'Port': ['ID=063b884b-9614-48ee-9ca9-0c68b3e5f35f,revision_number=3']} {{(pid=91646) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:45.699864 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-607f95c8-48de-4b82-8554-fd706f67790b None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.091s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:45.719692 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-070ff5da-300d-42b2-bdd7-d34399d17dbd admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:45.721542 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-c93632de-3407-4063-872d-3059b2ac2e2f admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=a1d94e15f8bd4d5b8d1adc8835ff314d&name=default HTTP/1.1" status: 200 len: 2975 time: 1.5767167 May 20 04:21:45.724355 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Routers {'5a3f69b9-fd58-499f-bec9-75215cd7f8fc'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:21:45.724982 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:21:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:21:45.725117 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 60804) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:45.753014 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Allocated IP 172.20.0.1 (64bcea8a-6d35-406b-a823-fa1185008174/203a3962-2282-4f51-ac92-c9695b5c0cfe/ac9495b7-5868-4104-85c9-3567f1350467) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:21:45.791763 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-ebbebe18-cf8a-499a-af30-507c3dac3c4b service neutron] 10.222.0.39 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.8004434 May 20 04:21:45.810205 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-c911b23c-0d16-4f9a-a17b-aca1a6d5cd08 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:21:45.810986 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-c911b23c-0d16-4f9a-a17b-aca1a6d5cd08 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Synchronizing usage tracker for tenant:0b29d684ade544028fa17828cde46d56 on resource:network {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:21:45.948844 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] Set router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:21:45.971101 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4b8c977c-21ce-459e-9f76-41fb3614aa21 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:45.977373 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-afc5000c-f4b7-435a-ad82-30efce6ea089 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=8560a3f4-1b5b-435f-a516-2e3cf15ad5b5,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:45.995138 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-070ff5da-300d-42b2-bdd7-d34399d17dbd admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:46.001642 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-afc5000c-f4b7-435a-ad82-30efce6ea089 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.419s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:46.020803 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4b8c977c-21ce-459e-9f76-41fb3614aa21 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:46.077296 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:46.123033 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-c911b23c-0d16-4f9a-a17b-aca1a6d5cd08 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Unset dirty status for tenant:0b29d684ade544028fa17828cde46d56 on resource:network {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:21:46.124223 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-c911b23c-0d16-4f9a-a17b-aca1a6d5cd08 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/networks?id=a009b619-8ab3-40e3-b5f3-9cc97ccd3415 HTTP/1.1" status: 200 len: 810 time: 0.7240896 May 20 04:21:46.128036 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:46.131280 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:46.164554 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:46.179587 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Transition to ACTIVE for port object ac9495b7-5868-4104-85c9-3567f1350467 will not be triggered until provisioned by entity DHCP. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:21:46.188645 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-c4ac934f-8254-49cf-ad53-c540b6355acc tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/quotas/0b29d684ade544028fa17828cde46d56 HTTP/1.1" status: 200 len: 375 time: 0.0177054 May 20 04:21:46.225969 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4b8c977c-21ce-459e-9f76-41fb3614aa21 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:46.244306 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-ec01db29-0c83-4c55-b0f5-c86215b08a36 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:46.251741 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-dd89236b-98ee-42d7-9ce3-d19d6167f166 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=0b29d684ade544028fa17828cde46d56&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0565720 May 20 04:21:46.269145 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4b8c977c-21ce-459e-9f76-41fb3614aa21 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d383c378-6b96-4b11-881b-1694db0caa3a,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:46.282769 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-4b8c977c-21ce-459e-9f76-41fb3614aa21 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.038s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:46.315941 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:46.319049 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-070ff5da-300d-42b2-bdd7-d34399d17dbd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:46.322585 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-b94d10f0-9cab-451b-a5b1-abea7ba400cf admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:46.357959 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:46.371628 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-070ff5da-300d-42b2-bdd7-d34399d17dbd admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=8560a3f4-1b5b-435f-a516-2e3cf15ad5b5,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:46.374481 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-070ff5da-300d-42b2-bdd7-d34399d17dbd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:46.422185 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-070ff5da-300d-42b2-bdd7-d34399d17dbd admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:46.423530 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-070ff5da-300d-42b2-bdd7-d34399d17dbd admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/8560a3f4-1b5b-435f-a516-2e3cf15ad5b5 HTTP/1.1" status: 204 len: 168 time: 0.8071198 May 20 04:21:46.434589 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Allocated IP 172.27.0.120 (712ce8c4-8feb-4719-bf13-e06992125cd9/e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5/68d41f86-efee-45dc-988a-b22c7179ec88) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:21:46.480008 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4b8c977c-21ce-459e-9f76-41fb3614aa21 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:46.483024 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-4b8c977c-21ce-459e-9f76-41fb3614aa21 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/d383c378-6b96-4b11-881b-1694db0caa3a HTTP/1.1" status: 204 len: 168 time: 0.7569597 May 20 04:21:46.484355 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Pushing event updated for resources: {'Port': ['ID=ac9495b7-5868-4104-85c9-3567f1350467,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:46.502353 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Network 64bcea8a-6d35-406b-a823-fa1185008174 is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:21:46.512058 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:21:46.528218 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:21:40Z', 'revision_number': 3, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port': {'id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:e4:9a:7c', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:39Z', 'updated_at': '2021-05-20T04:21:45Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:21:46.531125 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.194s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:46.531950 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:21:46.533178 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-8003132'] for router_interface, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:46.538106 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-beaee614-d226-4c7d-97a9-4d826f763c36 None None] The sync data for ids ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'] in dv-cinder-782405-9-iscsi is: [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:21:40Z', 'revision_number': 3, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port': {'id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:e4:9a:7c', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:39Z', 'updated_at': '2021-05-20T04:21:45Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}] {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:21:46.602349 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-36f1ca6c-bf0b-4b0f-a14a-cc8268016fe0 service neutron] http://10.222.0.39:9696/v2.0/extensions returned with HTTP 200 {{(pid=91643) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 20 04:21:46.609666 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-36f1ca6c-bf0b-4b0f-a14a-cc8268016fe0 service neutron] 10.222.0.39 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.1886165 May 20 04:21:46.613100 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ad93daaf-c491-435d-bb10-3c8bf8068f17 req-9b99afe2-3648-414d-bcb5-c0ba38708b38 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 210 time: 0.0798488 May 20 04:21:47.026236 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-132ac57e-3d4e-4b47-939c-718d5522b563 None None] Provisioning for port ac9495b7-5868-4104-85c9-3567f1350467 completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:21:47.038303 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-132ac57e-3d4e-4b47-939c-718d5522b563 None None] Provisioning complete for port ac9495b7-5868-4104-85c9-3567f1350467 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:21:47.038694 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-132ac57e-3d4e-4b47-939c-718d5522b563 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:47.173413 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:47.264081 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-132ac57e-3d4e-4b47-939c-718d5522b563 None None] Port ac9495b7-5868-4104-85c9-3567f1350467 cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:21:47.280405 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 063b884b-9614-48ee-9ca9-0c68b3e5f35f up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:21:47.307435 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:47.366131 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:47.367771 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:47.475965 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:21:47.485881 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:21:47.514232 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:21:47.610380 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port 063b884b-9614-48ee-9ca9-0c68b3e5f35f completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:21:47.616445 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 063b884b-9614-48ee-9ca9-0c68b3e5f35f triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:21:47.616818 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:47.845506 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:47.987357 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:47.991405 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-ec52a21f-2ca7-4a33-9b91-a32acb6c0943 service neutron] 10.222.0.39 "GET /v2.0/networks/a009b619-8ab3-40e3-b5f3-9cc97ccd3415?fields=segments HTTP/1.1" status: 200 len: 212 time: 1.3808615 May 20 04:21:48.016596 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port 063b884b-9614-48ee-9ca9-0c68b3e5f35f, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=063b884b-9614-48ee-9ca9-0c68b3e5f35f,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:21:48.028129 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:48.030854 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-fddf7d60-b533-4f61-be94-88c30852cf58 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:48.047918 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:48.058763 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:48.060005 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-7924897', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-8003163'] for router_interface, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:48.067008 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:48.177975 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=063b884b-9614-48ee-9ca9-0c68b3e5f35f,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:48.183500 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.153s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:48.226134 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4b41fcf8-219f-46eb-8934-9dcaed6e9157 req-abf61b9e-c033-4fa4-b656-f9b79550570f tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 210 time: 0.2188959 May 20 04:21:48.319626 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Pushing event updated for resources: {'Port': ['ID=ac9495b7-5868-4104-85c9-3567f1350467,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:48.338362 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.269s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:48.384707 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Pushing event updated for resources: {'Port': ['ID=68d41f86-efee-45dc-988a-b22c7179ec88,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:48.393249 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Network 64bcea8a-6d35-406b-a823-fa1185008174 is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:21:48.394519 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:21:48.413491 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 1.047s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:48.680939 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-c960a68d-cfc0-4000-a2c7-0ba43a393a5b service neutron] 10.222.0.39 "GET /v2.0/networks/a009b619-8ab3-40e3-b5f3-9cc97ccd3415?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.6128280 May 20 04:21:48.725856 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:48.805737 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91644) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:21:48.807684 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-8cb2cbd8-3b5f-4af9-b733-608b9419d88e None None] Provisioning complete for port ac9495b7-5868-4104-85c9-3567f1350467 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:21:48.808157 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-8cb2cbd8-3b5f-4af9-b733-608b9419d88e None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:48.964731 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-8cb2cbd8-3b5f-4af9-b733-608b9419d88e None None] Port ac9495b7-5868-4104-85c9-3567f1350467 cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:21:49.176974 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 063b884b-9614-48ee-9ca9-0c68b3e5f35f up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:21:49.240499 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:21:49.246636 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] 10.222.0.39 "PUT /v2.0/routers/5a3f69b9-fd58-499f-bec9-75215cd7f8fc/add_router_interface HTTP/1.1" status: 200 len: 503 time: 5.7447994 May 20 04:21:49.296906 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Sync routers for ids ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:21:49.315455 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 60968) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:49.337189 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Routers {'5a3f69b9-fd58-499f-bec9-75215cd7f8fc'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:21:49.338001 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:21:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:21:49.434873 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 063b884b-9614-48ee-9ca9-0c68b3e5f35f triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:21:49.435607 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:49.513718 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-561df2b0-83be-408f-b2ad-279bff8142d4 req-03a5881a-5245-4f99-b5f5-e5591c2872ce tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 210 time: 0.1339891 May 20 04:21:49.592608 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-209cb506-b199-4523-9dd2-e7fdad912c53 req-0cfc985f-1c86-46fa-bf5e-8bd54645c433 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 210 time: 0.0835888 May 20 04:21:49.600312 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.74', 56922) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:49.743751 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-8071551', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-8053391'] for router, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:49.796885 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Set router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:21:49.932720 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-ce674a04-0e03-404d-a6d4-38177b071838 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:21:49.934232 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-ce674a04-0e03-404d-a6d4-38177b071838 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.74 "GET /v2.0/networks?id=54fc8d38-193b-49c4-b873-c5353723791f HTTP/1.1" status: 200 len: 815 time: 0.3328431 May 20 04:21:49.980806 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-57edf988-28ff-48f1-85ab-a8390c861654 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:50.046084 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-ec91e9d5-a4de-4741-a4c5-5872927e6cf4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Request body: {'port': {'device_id': 'e94a8b2b-8a29-45d4-9edb-63137be69ce4', 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f', 'admin_state_up': True, 'tenant_id': '51fcbc78284541db832e4780e76158af'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:21:50.089089 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-57edf988-28ff-48f1-85ab-a8390c861654 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:50.095537 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-57edf988-28ff-48f1-85ab-a8390c861654 admin admin] Usage tracker for resource:security_group_rule and tenant:ad4340e269c249aba9f989ed8036c90b is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:50.098269 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Router be02e160-6134-4b60-b833-5a659cc82c44 is scheduled to L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 with binding_index 1 {{(pid=91643) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 20 04:21:50.106744 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-57edf988-28ff-48f1-85ab-a8390c861654 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:50.119201 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-57edf988-28ff-48f1-85ab-a8390c861654 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:50.358081 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port': {'id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:e4:9a:7c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:39Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'ac9495b7-5868-4104-85c9-3567f1350467', 'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'mac_address': 'fa:16:3e:12:5b:da', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_interface', 'standard_attr_id': 82, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:45Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 2, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'subnets': [{'id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:21:50.363262 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Checking router: 5a3f69b9-fd58-499f-bec9-75215cd7f8fc for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:21:50.363945 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Updating router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc port ac9495b7-5868-4104-85c9-3567f1350467 binding host dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 20 04:21:50.439620 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:50.696534 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:21:50.709258 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Sync routers for ids ['be02e160-6134-4b60-b833-5a659cc82c44'] in dv-cinder-782405-9-iscsi {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:21:50.720967 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-57edf988-28ff-48f1-85ab-a8390c861654 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:50.723045 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-ec91e9d5-a4de-4741-a4c5-5872927e6cf4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:50.730995 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-ec91e9d5-a4de-4741-a4c5-5872927e6cf4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Usage tracker for resource:port and tenant:51fcbc78284541db832e4780e76158af is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:50.739879 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Routers {'be02e160-6134-4b60-b833-5a659cc82c44'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:21:50.742233 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-ec91e9d5-a4de-4741-a4c5-5872927e6cf4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Quota usage for port was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:50.743095 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:21:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['be02e160-6134-4b60-b833-5a659cc82c44'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:21:50.758190 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Notify callbacks [] for router, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:50.760025 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] 10.222.0.39 "POST /v2.0/routers HTTP/1.1" status: 201 len: 931 time: 7.2872524 May 20 04:21:50.762716 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 32828) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:50.771956 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-57edf988-28ff-48f1-85ab-a8390c861654 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:50.809087 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-ec91e9d5-a4de-4741-a4c5-5872927e6cf4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:50.822534 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-c93632de-3407-4063-872d-3059b2ac2e2f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:50.895839 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-57edf988-28ff-48f1-85ab-a8390c861654 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=63bf6d45-87ac-4d64-b602-c551ef807a38,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:50.936001 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-57edf988-28ff-48f1-85ab-a8390c861654 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.115s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:50.992536 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-57edf988-28ff-48f1-85ab-a8390c861654 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=ad4340e269c249aba9f989ed8036c90b&name=default HTTP/1.1" status: 200 len: 2975 time: 1.6747653 May 20 04:21:50.993689 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-ec91e9d5-a4de-4741-a4c5-5872927e6cf4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Made reservation on behalf of 51fcbc78284541db832e4780e76158af for: {'port': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:21:50.994265 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 32842) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:51.005538 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-ec91e9d5-a4de-4741-a4c5-5872927e6cf4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:51.194872 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 32852) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:51.206038 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-4259269b-8b1c-444e-ad15-e7f3cf69ff00 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Request body: {'security_group': {'name': 'tempest-securitygroup--635106175', 'description': 'tempest-description--1277769102'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:21:51.221504 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Set router be02e160-6134-4b60-b833-5a659cc82c44 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:21:51.427698 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Notify callbacks [] for port, precommit_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:51.443936 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Transition to ACTIVE for port object ac9495b7-5868-4104-85c9-3567f1350467 will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:21:51.452003 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-4259269b-8b1c-444e-ad15-e7f3cf69ff00 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:51.454200 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:51.462447 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:51.484370 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-4259269b-8b1c-444e-ad15-e7f3cf69ff00 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Usage tracker for resource:security_group and tenant:9fafdf3a29024ac588e5b24719735677 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:51.491864 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b0fd0abd-bfe1-41b5-94bc-9a7ddc06871e req-b7366728-bb4f-41bf-b84e-1c49081c49f1 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 210 time: 0.3500116 May 20 04:21:51.514583 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-4259269b-8b1c-444e-ad15-e7f3cf69ff00 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:51.523971 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a6f50bec-7794-4c8c-b3fc-b4b96d89c277 req-6cfcf3be-3dd7-4ccd-abda-0edff5463ce5 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 210 time: 0.1640215 May 20 04:21:51.550996 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7888bd43-9750-4a9c-90c4-99cf985528f1 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:51.572769 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-4259269b-8b1c-444e-ad15-e7f3cf69ff00 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:51.616019 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-ec91e9d5-a4de-4741-a4c5-5872927e6cf4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Allocated IP 172.20.0.14 (54fc8d38-193b-49c4-b873-c5353723791f/0f918085-8d33-4e36-a59e-7f270a2b1874/849868be-af51-4967-9a6c-93d23cecb450) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:21:51.630509 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4259269b-8b1c-444e-ad15-e7f3cf69ff00 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:51.722186 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-4259269b-8b1c-444e-ad15-e7f3cf69ff00 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:51.763079 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-4259269b-8b1c-444e-ad15-e7f3cf69ff00 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Usage tracker for resource:security_group_rule and tenant:9fafdf3a29024ac588e5b24719735677 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:51.765900 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.74', 56931) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:51.816161 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-4259269b-8b1c-444e-ad15-e7f3cf69ff00 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:51.940634 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-4259269b-8b1c-444e-ad15-e7f3cf69ff00 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:51.964737 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7888bd43-9750-4a9c-90c4-99cf985528f1 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:51.984722 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'be02e160-6134-4b60-b833-5a659cc82c44', 'name': 'tempest-ServerRescueNegativeTestJSON-1026520542-router', 'tenant_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.120'}], 'enable_snat': True}, 'gw_port_id': '68d41f86-efee-45dc-988a-b22c7179ec88', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:43Z', 'updated_at': '2021-05-20T04:21:48Z', 'revision_number': 3, 'project_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'gw_port': {'id': '68d41f86-efee-45dc-988a-b22c7179ec88', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:65:dc:1e', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'be02e160-6134-4b60-b833-5a659cc82c44', 'device_owner': 'network:router_gateway', 'standard_attr_id': 83, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.120', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:46Z', 'updated_at': '2021-05-20T04:21:46Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:21:51.993087 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Checking router: be02e160-6134-4b60-b833-5a659cc82c44 for host: dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:21:51.993334 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Updating router be02e160-6134-4b60-b833-5a659cc82c44 port 68d41f86-efee-45dc-988a-b22c7179ec88 binding host dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 20 04:21:52.151970 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8eed0b07-ca9a-4558-ad1c-f21ff862419c None None] Pushing event updated for resources: {'Port': ['ID=ac9495b7-5868-4104-85c9-3567f1350467,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:52.200993 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:52.204770 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-8eed0b07-ca9a-4558-ad1c-f21ff862419c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.742s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:52.240796 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Attempting to bind port ac9495b7-5868-4104-85c9-3567f1350467 on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:21:52.247461 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Attempting to bind port ac9495b7-5868-4104-85c9-3567f1350467 by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '9d1b8515-c35d-48e6-86ad-4eff412e924a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 847, 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:21:52.247977 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Attempting to bind port ac9495b7-5868-4104-85c9-3567f1350467 on network 64bcea8a-6d35-406b-a823-fa1185008174 {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:21:52.262418 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 21, 45), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:21:52.264139 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Checking segment: {'id': '9d1b8515-c35d-48e6-86ad-4eff412e924a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 847, 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:21:52.266886 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:52.267491 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Bound using segment: {'id': '9d1b8515-c35d-48e6-86ad-4eff412e924a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 847, 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:21:52.268117 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Bound port: ac9495b7-5868-4104-85c9-3567f1350467, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '9d1b8515-c35d-48e6-86ad-4eff412e924a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 847, 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:21:52.270422 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:52.463942 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] For port ac9495b7-5868-4104-85c9-3567f1350467, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:21:52.467967 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] For port ac9495b7-5868-4104-85c9-3567f1350467, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=ac9495b7-5868-4104-85c9-3567f1350467,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:21:52.472660 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:52.485636 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:52.804049 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 32902) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:52.805441 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Routers {'5a3f69b9-fd58-499f-bec9-75215cd7f8fc'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:21:52.808348 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:21:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:21:53.024540 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8eed0b07-ca9a-4558-ad1c-f21ff862419c None None] Pushing event updated for resources: {'Port': ['ID=ac9495b7-5868-4104-85c9-3567f1350467,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:53.038172 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-8eed0b07-ca9a-4558-ad1c-f21ff862419c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.552s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:53.183696 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-82351f4a-18bb-4b46-b350-bee7a1ab16c5 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:21:53.184717 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7888bd43-9750-4a9c-90c4-99cf985528f1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:53.186165 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-82351f4a-18bb-4b46-b350-bee7a1ab16c5 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.74 "GET /v2.0/networks?id=a009b619-8ab3-40e3-b5f3-9cc97ccd3415 HTTP/1.1" status: 200 len: 810 time: 1.4191389 May 20 04:21:53.214811 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-1c784839-874f-48d4-9897-af7f8122b6ae tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Request body: {'port': {'device_id': 'bea4be17-559d-407e-9b61-ff2cb94317dc', 'network_id': 'a009b619-8ab3-40e3-b5f3-9cc97ccd3415', 'admin_state_up': True, 'tenant_id': '0b29d684ade544028fa17828cde46d56'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:21:53.232709 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-46aefa71-5569-4520-9967-2cd870a543c7 req-089b9885-5ae2-4fad-9a40-e794d2233445 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 210 time: 0.1783209 May 20 04:21:53.293354 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Notify callbacks [] for port, precommit_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:53.321313 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.provisioning_blocks [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Transition to ACTIVE for port object 68d41f86-efee-45dc-988a-b22c7179ec88 will not be triggered until provisioned by entity L2. {{(pid=91646) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:21:53.335348 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:53.345858 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:53.368759 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] Set router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:21:53.407392 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Attempting to bind port 68d41f86-efee-45dc-988a-b22c7179ec88 on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:21:53.407783 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Attempting to bind port 68d41f86-efee-45dc-988a-b22c7179ec88 by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}] {{(pid=91646) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:21:53.408125 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Attempting to bind port 68d41f86-efee-45dc-988a-b22c7179ec88 on network 712ce8c4-8feb-4719-bf13-e06992125cd9 {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:21:53.408293 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b353e2ee-0460-4b59-b1e3-4f699d365142 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:53.429538 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 21, 45), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:21:53.429976 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Checking segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91646) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:21:53.431145 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:53.431580 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Bound using segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:21:53.432243 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Bound port: 68d41f86-efee-45dc-988a-b22c7179ec88, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}}] {{(pid=91646) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:21:53.432643 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:53.439090 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-afc5000c-f4b7-435a-ad82-30efce6ea089 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:53.453115 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4259269b-8b1c-444e-ad15-e7f3cf69ff00 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:53.465642 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-b353e2ee-0460-4b59-b1e3-4f699d365142 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:53.469290 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4259269b-8b1c-444e-ad15-e7f3cf69ff00 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:53.476599 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-ec91e9d5-a4de-4741-a4c5-5872927e6cf4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:53.485437 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-b353e2ee-0460-4b59-b1e3-4f699d365142 admin admin] Usage tracker for resource:security_group_rule and tenant:cd390eb8e00c4e2d86d55cb9b4bc61ab is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:53.495466 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c393b33f-2a81-475d-994b-533acbff3d2f None None] Pushing event updated for resources: {'Port': ['ID=68d41f86-efee-45dc-988a-b22c7179ec88,revision_number=2']} {{(pid=91646) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:53.498710 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-b353e2ee-0460-4b59-b1e3-4f699d365142 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:53.500173 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-c393b33f-2a81-475d-994b-533acbff3d2f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.154s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:53.501889 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:53.546133 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.db [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] For port 68d41f86-efee-45dc-988a-b22c7179ec88, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91646) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:21:53.555437 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.db [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] For port 68d41f86-efee-45dc-988a-b22c7179ec88, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=68d41f86-efee-45dc-988a-b22c7179ec88,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91646) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:21:53.561813 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4259269b-8b1c-444e-ad15-e7f3cf69ff00 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Pushing event updated for resources: {'SecurityGroup': ['ID=202eefda-65f8-4304-8243-021366a25be9,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:53.563786 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:53.570969 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-b353e2ee-0460-4b59-b1e3-4f699d365142 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:53.583969 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-4259269b-8b1c-444e-ad15-e7f3cf69ff00 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.082s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:53.588180 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-c393b33f-2a81-475d-994b-533acbff3d2f None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:53.598677 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7888bd43-9750-4a9c-90c4-99cf985528f1 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=63bf6d45-87ac-4d64-b602-c551ef807a38,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:53.612392 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-7888bd43-9750-4a9c-90c4-99cf985528f1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.172s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:53.676194 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Routers {'be02e160-6134-4b60-b833-5a659cc82c44'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:21:53.676846 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:21:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['be02e160-6134-4b60-b833-5a659cc82c44'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:21:53.700082 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-1c784839-874f-48d4-9897-af7f8122b6ae tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:53.716145 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-ec91e9d5-a4de-4741-a4c5-5872927e6cf4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:53.735627 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d50165db-c059-43b7-af36-c90794b334ea req-c2f4cc6e-d40e-4431-864a-6cbd2ae94abd tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 210 time: 0.6822035 May 20 04:21:53.775915 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7888bd43-9750-4a9c-90c4-99cf985528f1 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:53.777869 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-7888bd43-9750-4a9c-90c4-99cf985528f1 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/63bf6d45-87ac-4d64-b602-c551ef807a38 HTTP/1.1" status: 204 len: 168 time: 2.7824745 May 20 04:21:53.783584 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:53.898118 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b353e2ee-0460-4b59-b1e3-4f699d365142 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:53.903751 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-1c784839-874f-48d4-9897-af7f8122b6ae tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Usage tracker for resource:port and tenant:0b29d684ade544028fa17828cde46d56 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:53.914563 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4259269b-8b1c-444e-ad15-e7f3cf69ff00 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:53.920756 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-4259269b-8b1c-444e-ad15-e7f3cf69ff00 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1777 time: 2.7245245 May 20 04:21:53.924215 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-1c784839-874f-48d4-9897-af7f8122b6ae tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Quota usage for port was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:53.932192 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 32918) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:53.940681 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b353e2ee-0460-4b59-b1e3-4f699d365142 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:53.954349 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-fb35603a-72d2-46ca-8fcf-9aefb8321c3e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Request body: {'security_group_rule': {'security_group_id': '202eefda-65f8-4304-8243-021366a25be9', 'protocol': 'tcp', 'ethertype': 'IPv4', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:21:53.958385 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-1c784839-874f-48d4-9897-af7f8122b6ae tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:53.980269 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-50ff10a4-0e23-47ab-b8dd-9b1450579cf2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:53.990480 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-fb35603a-72d2-46ca-8fcf-9aefb8321c3e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:54.028303 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-fb35603a-72d2-46ca-8fcf-9aefb8321c3e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Usage tracker for resource:security_group_rule and tenant:9fafdf3a29024ac588e5b24719735677 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:54.047330 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-fb35603a-72d2-46ca-8fcf-9aefb8321c3e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:54.081085 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c393b33f-2a81-475d-994b-533acbff3d2f None None] Pushing event updated for resources: {'Port': ['ID=68d41f86-efee-45dc-988a-b22c7179ec88,revision_number=3']} {{(pid=91646) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:54.086055 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-c393b33f-2a81-475d-994b-533acbff3d2f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.498s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:54.102171 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] Set router be02e160-6134-4b60-b833-5a659cc82c44 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:21:54.157164 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-fb35603a-72d2-46ca-8fcf-9aefb8321c3e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:54.218187 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-ec91e9d5-a4de-4741-a4c5-5872927e6cf4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Network 54fc8d38-193b-49c4-b873-c5353723791f is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:21:54.280769 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-ec91e9d5-a4de-4741-a4c5-5872927e6cf4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:21:54.294267 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-ec91e9d5-a4de-4741-a4c5-5872927e6cf4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:21:54.303571 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-ec91e9d5-a4de-4741-a4c5-5872927e6cf4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:21:54.332655 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-ec91e9d5-a4de-4741-a4c5-5872927e6cf4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:54.358183 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-fb35603a-72d2-46ca-8fcf-9aefb8321c3e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:54.394829 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port': {'id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:e4:9a:7c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:39Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'ac9495b7-5868-4104-85c9-3567f1350467', 'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'mac_address': 'fa:16:3e:12:5b:da', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_interface', 'standard_attr_id': 82, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:45Z', 'updated_at': '2021-05-20T04:21:52Z', 'revision_number': 4, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'subnets': [{'id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:21:54.410972 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-bab3e9e8-2060-4573-934b-29ba579d55b6 None None] The sync data for ids ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'] in dv-cinder-782405-9-iscsi is: [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port': {'id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:e4:9a:7c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:39Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'ac9495b7-5868-4104-85c9-3567f1350467', 'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'mac_address': 'fa:16:3e:12:5b:da', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_interface', 'standard_attr_id': 82, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:45Z', 'updated_at': '2021-05-20T04:21:52Z', 'revision_number': 4, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'subnets': [{'id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:21:54.423650 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-1c784839-874f-48d4-9897-af7f8122b6ae tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Made reservation on behalf of 0b29d684ade544028fa17828cde46d56 for: {'port': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:21:54.431474 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-1c784839-874f-48d4-9897-af7f8122b6ae tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:54.527979 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-ec91e9d5-a4de-4741-a4c5-5872927e6cf4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 4.4928277 May 20 04:21:54.551463 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-b353e2ee-0460-4b59-b1e3-4f699d365142 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=cd390eb8e00c4e2d86d55cb9b4bc61ab&name=default HTTP/1.1" status: 200 len: 2975 time: 1.7309682 May 20 04:21:54.553699 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 32940) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:54.554345 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ec91e9d5-a4de-4741-a4c5-5872927e6cf4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Pushing event updated for resources: {'Port': ['ID=849868be-af51-4967-9a6c-93d23cecb450,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:54.564300 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-ec91e9d5-a4de-4741-a4c5-5872927e6cf4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.585s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:54.569110 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.428s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:54.607890 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-4cb232a3-70c4-485b-b30e-bb3eb8b9fde3 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] http://10.222.0.39:9696/v2.0/extensions returned with HTTP 200 {{(pid=91644) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 20 04:21:54.608792 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-4cb232a3-70c4-485b-b30e-bb3eb8b9fde3 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.74 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0413909 May 20 04:21:54.792049 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-400caff9-650c-430c-b091-8c9103b429bf req-9215d9bd-92b3-44b4-af1f-adbb897b8990 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 210 time: 0.1997910 May 20 04:21:54.810166 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b353e2ee-0460-4b59-b1e3-4f699d365142 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e42e2fdf-bf61-49d3-aa91-c61caa50318e,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:54.846811 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Allocated IP 172.20.0.1 (36b37213-8439-447d-9eba-a78baf7c698e/b25ae9c8-e11d-45ee-bde2-332cc1538602/29b05a85-1060-41b1-aff1-48c1783a174f) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:21:54.856252 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-b353e2ee-0460-4b59-b1e3-4f699d365142 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.287s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:54.939343 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-fb35603a-72d2-46ca-8fcf-9aefb8321c3e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:54.957376 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-fb35603a-72d2-46ca-8fcf-9aefb8321c3e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:55.006299 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-2a05e45e-0628-4cf3-8431-3e5337def7de admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:55.035142 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:55.058914 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-2a05e45e-0628-4cf3-8431-3e5337def7de admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:55.080723 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fb35603a-72d2-46ca-8fcf-9aefb8321c3e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=96473866-eabe-4734-9c66-66aa2c9682b0,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:55.105100 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'be02e160-6134-4b60-b833-5a659cc82c44', 'name': 'tempest-ServerRescueNegativeTestJSON-1026520542-router', 'tenant_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.120'}], 'enable_snat': True}, 'gw_port_id': '68d41f86-efee-45dc-988a-b22c7179ec88', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:43Z', 'updated_at': '2021-05-20T04:21:48Z', 'revision_number': 3, 'project_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'gw_port': {'id': '68d41f86-efee-45dc-988a-b22c7179ec88', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:65:dc:1e', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'be02e160-6134-4b60-b833-5a659cc82c44', 'device_owner': 'network:router_gateway', 'standard_attr_id': 83, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.120', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:46Z', 'updated_at': '2021-05-20T04:21:53Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:21:55.114769 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-1c784839-874f-48d4-9897-af7f8122b6ae tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Allocated IP 172.20.0.2 (a009b619-8ab3-40e3-b5f3-9cc97ccd3415/b9f2dbbb-b724-44b9-878b-2267c4d95aee/efb81444-f373-4e9c-a0aa-e3270bd4b980) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:21:55.131243 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4d81472e-fb89-4d90-8dcc-7226d1fe8bb7 None None] The sync data for ids ['be02e160-6134-4b60-b833-5a659cc82c44'] in dv-cinder-782405-9-iscsi is: [{'id': 'be02e160-6134-4b60-b833-5a659cc82c44', 'name': 'tempest-ServerRescueNegativeTestJSON-1026520542-router', 'tenant_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.120'}], 'enable_snat': True}, 'gw_port_id': '68d41f86-efee-45dc-988a-b22c7179ec88', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:43Z', 'updated_at': '2021-05-20T04:21:48Z', 'revision_number': 3, 'project_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'gw_port': {'id': '68d41f86-efee-45dc-988a-b22c7179ec88', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:65:dc:1e', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'be02e160-6134-4b60-b833-5a659cc82c44', 'device_owner': 'network:router_gateway', 'standard_attr_id': 83, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.120', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:46Z', 'updated_at': '2021-05-20T04:21:53Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}] {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:21:55.269060 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device ac9495b7-5868-4104-85c9-3567f1350467 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:21:55.370562 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-2a05e45e-0628-4cf3-8431-3e5337def7de admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:55.435196 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:55.451804 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port ac9495b7-5868-4104-85c9-3567f1350467 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:21:55.463393 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port ac9495b7-5868-4104-85c9-3567f1350467 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:21:55.463772 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:55.470275 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Request body: {'port': {'device_id': 'e94a8b2b-8a29-45d4-9edb-63137be69ce4', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:21:55.604198 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:55.689957 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-fb35603a-72d2-46ca-8fcf-9aefb8321c3e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.655s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:55.756021 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-fb35603a-72d2-46ca-8fcf-9aefb8321c3e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:55.760692 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-fb35603a-72d2-46ca-8fcf-9aefb8321c3e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 1.8269494 May 20 04:21:55.765140 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 32966) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:55.768886 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:55.799596 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-33908b5a-4e53-4990-91b8-22e0d17bbac8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Request body: {'security_group_rule': {'security_group_id': '202eefda-65f8-4304-8243-021366a25be9', 'protocol': 'icmp', 'ethertype': 'IPv4', 'direction': 'ingress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:21:55.834260 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-33908b5a-4e53-4990-91b8-22e0d17bbac8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:55.851244 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-33908b5a-4e53-4990-91b8-22e0d17bbac8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Usage tracker for resource:security_group_rule and tenant:9fafdf3a29024ac588e5b24719735677 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:55.860565 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-33908b5a-4e53-4990-91b8-22e0d17bbac8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:55.886593 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-33908b5a-4e53-4990-91b8-22e0d17bbac8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:55.895269 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:55.946339 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 32978) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:55.952261 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:56.016314 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port ac9495b7-5868-4104-85c9-3567f1350467, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=ac9495b7-5868-4104-85c9-3567f1350467,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:21:56.026200 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:56.026658 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2a05e45e-0628-4cf3-8431-3e5337def7de admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e42e2fdf-bf61-49d3-aa91-c61caa50318e,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:56.039505 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-2a05e45e-0628-4cf3-8431-3e5337def7de admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.604s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:56.041909 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.001s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:56.104820 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Network 36b37213-8439-447d-9eba-a78baf7c698e is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:21:56.181719 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-2a05e45e-0628-4cf3-8431-3e5337def7de admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:56.182903 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:21:56.182903 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:21:56.182903 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-8003132'] for router_interface, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:56.182903 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-33908b5a-4e53-4990-91b8-22e0d17bbac8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:56.192110 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-2a05e45e-0628-4cf3-8431-3e5337def7de admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/e42e2fdf-bf61-49d3-aa91-c61caa50318e HTTP/1.1" status: 204 len: 168 time: 1.6274102 May 20 04:21:56.269869 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=ac9495b7-5868-4104-85c9-3567f1350467,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:56.274529 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.233s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:56.482077 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-33908b5a-4e53-4990-91b8-22e0d17bbac8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:56.490686 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-358e62e5-1830-44f8-80b9-9bc02d719445 req-42acd368-3963-4490-b395-78381e3beb64 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 210 time: 0.2072146 May 20 04:21:56.550134 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-33908b5a-4e53-4990-91b8-22e0d17bbac8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:56.603669 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Pushing event updated for resources: {'Port': ['ID=29b05a85-1060-41b1-aff1-48c1783a174f,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:56.614132 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-1c784839-874f-48d4-9897-af7f8122b6ae tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:56.653583 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-76edaf36-457a-4b67-ad47-c8b27bbbdc98 req-f42fbb89-527c-481a-a143-0c2e74d83ab2 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:21:56.668046 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-76edaf36-457a-4b67-ad47-c8b27bbbdc98 req-f42fbb89-527c-481a-a143-0c2e74d83ab2 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 932 time: 1.5882988 May 20 04:21:56.672696 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.720s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:56.675866 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:56.692027 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-fb35603a-72d2-46ca-8fcf-9aefb8321c3e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.104s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:56.776256 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Request body: {'network': {'name': 'tempest-EncryptedVolumesExtendAttachedTest-2048268485-network', 'tenant_id': '610f82a3f8c9479b83711aad8d2a1ca5'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:21:56.785837 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 32998) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:56.821602 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 33004) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:56.823054 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:56.836243 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-33908b5a-4e53-4990-91b8-22e0d17bbac8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=bcea710f-fca1-4886-b7c0-065d4a6152f6,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:56.856075 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-33908b5a-4e53-4990-91b8-22e0d17bbac8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.167s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:56.856830 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-33908b5a-4e53-4990-91b8-22e0d17bbac8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:56.867798 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-33908b5a-4e53-4990-91b8-22e0d17bbac8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 1.0939853 May 20 04:21:56.881759 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Usage tracker for resource:network and tenant:610f82a3f8c9479b83711aad8d2a1ca5 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:56.884212 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 33008) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:56.907752 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:56.917231 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-59b2047d-8f1f-4db1-b8e4-90f31d340654 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:56.944393 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Request body: {'floatingip': {'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:21:56.957590 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:56.961198 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-76edaf36-457a-4b67-ad47-c8b27bbbdc98 req-e5f227b2-09e9-4102-9806-ed9409cdd38e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Synchronizing usage tracker for tenant:51fcbc78284541db832e4780e76158af on resource:security_group {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:21:56.982781 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:57.015498 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Usage tracker for resource:floatingip and tenant:9fafdf3a29024ac588e5b24719735677 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:57.025672 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:57.029561 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Made reservation on behalf of 610f82a3f8c9479b83711aad8d2a1ca5 for: {'network': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:21:57.031826 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-585852'] for network, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:57.052950 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:57.056320 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-59b2047d-8f1f-4db1-b8e4-90f31d340654 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:57.057738 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:57.070004 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-59b2047d-8f1f-4db1-b8e4-90f31d340654 admin admin] Usage tracker for resource:security_group_rule and tenant:df59f2cd6b2a42b5b04611517248e11e is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:57.074170 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-59b2047d-8f1f-4db1-b8e4-90f31d340654 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:57.089557 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-59b2047d-8f1f-4db1-b8e4-90f31d340654 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:57.115355 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:57.142452 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-1c784839-874f-48d4-9897-af7f8122b6ae tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:57.196997 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:57.243437 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Usage tracker for resource:security_group_rule and tenant:610f82a3f8c9479b83711aad8d2a1ca5 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:57.247397 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:57.269232 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for floatingip, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:57.305285 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:57.346728 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-1c784839-874f-48d4-9897-af7f8122b6ae tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Network a009b619-8ab3-40e3-b5f3-9cc97ccd3415 is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:21:57.380108 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 68d41f86-efee-45dc-988a-b22c7179ec88 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:21:57.406720 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-1c784839-874f-48d4-9897-af7f8122b6ae tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:21:57.422391 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-db98a709-3ab7-4fab-a7ba-f968caf630b9 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:57.437201 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-1c784839-874f-48d4-9897-af7f8122b6ae tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:21:57.448969 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-1c784839-874f-48d4-9897-af7f8122b6ae tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:21:57.451663 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-1c784839-874f-48d4-9897-af7f8122b6ae tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:57.464335 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1c784839-874f-48d4-9897-af7f8122b6ae tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Pushing event updated for resources: {'Port': ['ID=efb81444-f373-4e9c-a0aa-e3270bd4b980,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:57.492049 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:57.512987 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-1c784839-874f-48d4-9897-af7f8122b6ae tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.316s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:57.527312 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port 68d41f86-efee-45dc-988a-b22c7179ec88 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:21:57.536827 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 68d41f86-efee-45dc-988a-b22c7179ec88 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:21:57.538020 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:57.543651 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-db98a709-3ab7-4fab-a7ba-f968caf630b9 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:21:57.556108 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-db98a709-3ab7-4fab-a7ba-f968caf630b9 admin admin] Usage tracker for resource:security_group_rule and tenant:25837feb29ab44b99a136d446cecfb77 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:21:57.561591 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-db98a709-3ab7-4fab-a7ba-f968caf630b9 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:21:57.565644 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-1c784839-874f-48d4-9897-af7f8122b6ae tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 4.3563626 May 20 04:21:57.576988 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-db98a709-3ab7-4fab-a7ba-f968caf630b9 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:21:57.619562 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-59b2047d-8f1f-4db1-b8e4-90f31d340654 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:57.632778 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-59b2047d-8f1f-4db1-b8e4-90f31d340654 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:57.651938 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-4b8c977c-21ce-459e-9f76-41fb3614aa21 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:57.762959 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.db.resource_extend [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] It took 0.12 seconds to run function 'neutron_lib.db.resource_extend.apply_funcs' {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py:386}} May 20 04:21:57.763910 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-db98a709-3ab7-4fab-a7ba-f968caf630b9 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:57.791427 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-59b2047d-8f1f-4db1-b8e4-90f31d340654 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=6aecd93a-227b-4036-80ac-c177f35cc784,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:57.802026 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-db98a709-3ab7-4fab-a7ba-f968caf630b9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:57.841489 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-59b2047d-8f1f-4db1-b8e4-90f31d340654 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.190s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:57.847830 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-59b2047d-8f1f-4db1-b8e4-90f31d340654 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=df59f2cd6b2a42b5b04611517248e11e&name=default HTTP/1.1" status: 200 len: 2979 time: 1.0148880 May 20 04:21:57.848484 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:57.875520 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 33020) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:57.882678 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port 68d41f86-efee-45dc-988a-b22c7179ec88, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=68d41f86-efee-45dc-988a-b22c7179ec88,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:21:57.902439 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:57.904127 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device ac9495b7-5868-4104-85c9-3567f1350467 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:21:57.935563 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 849868be-af51-4967-9a6c-93d23cecb450 {{(pid=91644) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:21:57.950286 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:57.971580 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-803f2476-d3a1-4a4c-a276-a15b23c2539c admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:57.983259 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-070ff5da-300d-42b2-bdd7-d34399d17dbd admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:57.990831 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:57.999554 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Transition to ACTIVE for port object 849868be-af51-4967-9a6c-93d23cecb450 will not be triggered until provisioned by entity L2. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:21:58.039307 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:58.042311 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-803f2476-d3a1-4a4c-a276-a15b23c2539c admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:58.072128 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:58.157188 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=68d41f86-efee-45dc-988a-b22c7179ec88,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:58.161211 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.170s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:58.170493 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port ac9495b7-5868-4104-85c9-3567f1350467 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:21:58.170877 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:58.222766 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-db98a709-3ab7-4fab-a7ba-f968caf630b9 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=25837feb29ab44b99a136d446cecfb77&name=default HTTP/1.1" status: 200 len: 2979 time: 1.4322696 May 20 04:21:58.234070 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 33028) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:21:58.408803 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Pushing event updated for resources: {'Port': ['ID=849868be-af51-4967-9a6c-93d23cecb450,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:58.416542 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.344s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:58.442563 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Attempting to bind port 849868be-af51-4967-9a6c-93d23cecb450 on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:21:58.442919 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Attempting to bind port 849868be-af51-4967-9a6c-93d23cecb450 by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': '6fe99325-a72c-48bf-abc7-9bbcc4b753ff', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 836, 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f'}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:21:58.443276 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Attempting to bind port 849868be-af51-4967-9a6c-93d23cecb450 on network 54fc8d38-193b-49c4-b873-c5353723791f {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:21:58.446561 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-db98a709-3ab7-4fab-a7ba-f968caf630b9 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=931085ef-a427-4564-9d2e-443d5a6a21a9,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:58.449608 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Port 849868be-af51-4967-9a6c-93d23cecb450 on network 54fc8d38-193b-49c4-b873-c5353723791f not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:21:58.450041 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Attempting to bind port 849868be-af51-4967-9a6c-93d23cecb450 on network 54fc8d38-193b-49c4-b873-c5353723791f {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:21:58.457002 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 21, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:21:58.457645 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Checking segment: {'id': '6fe99325-a72c-48bf-abc7-9bbcc4b753ff', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 836, 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:21:58.458316 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Bound using segment: {'id': '6fe99325-a72c-48bf-abc7-9bbcc4b753ff', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 836, 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f'} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:21:58.458995 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Bound port: 849868be-af51-4967-9a6c-93d23cecb450, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '6fe99325-a72c-48bf-abc7-9bbcc4b753ff', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 836, 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f'}}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:21:58.459446 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:58.479780 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-803f2476-d3a1-4a4c-a276-a15b23c2539c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:58.545225 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-db98a709-3ab7-4fab-a7ba-f968caf630b9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.564s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:58.551794 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-57edf988-28ff-48f1-85ab-a8390c861654 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:58.588234 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-803f2476-d3a1-4a4c-a276-a15b23c2539c admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=6aecd93a-227b-4036-80ac-c177f35cc784,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:58.594855 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-803f2476-d3a1-4a4c-a276-a15b23c2539c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:58.598901 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-803f2476-d3a1-4a4c-a276-a15b23c2539c admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:58.602612 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-803f2476-d3a1-4a4c-a276-a15b23c2539c admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/6aecd93a-227b-4036-80ac-c177f35cc784 HTTP/1.1" status: 204 len: 168 time: 0.7236466 May 20 04:21:58.620021 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:58.631529 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] For port 849868be-af51-4967-9a6c-93d23cecb450, host hv-cinder-78240, cleared binding levels {{(pid=91644) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:21:58.638691 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] For port 849868be-af51-4967-9a6c-93d23cecb450, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=849868be-af51-4967-9a6c-93d23cecb450,segment=NetworkSegment(6fe99325-a72c-48bf-abc7-9bbcc4b753ff),segment_id=6fe99325-a72c-48bf-abc7-9bbcc4b753ff)] {{(pid=91644) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:21:58.645848 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-202e1a79-0de0-4319-b075-d26e78b15aa7 req-7e64c6f9-5dad-4f62-8fe7-9e81efbb3e0e tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:21:58.648106 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-202e1a79-0de0-4319-b075-d26e78b15aa7 req-7e64c6f9-5dad-4f62-8fe7-9e81efbb3e0e tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 931 time: 0.8774614 May 20 04:21:58.653779 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:58.661922 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:58.688462 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:58.702910 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-7888bd43-9750-4a9c-90c4-99cf985528f1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:58.730472 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-76edaf36-457a-4b67-ad47-c8b27bbbdc98 req-e5f227b2-09e9-4102-9806-ed9409cdd38e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Unset dirty status for tenant:51fcbc78284541db832e4780e76158af on resource:security_group {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:21:58.737442 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-76edaf36-457a-4b67-ad47-c8b27bbbdc98 req-e5f227b2-09e9-4102-9806-ed9409cdd38e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 2.0679102 May 20 04:21:58.748619 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f59e3df7-daf1-48c8-8662-386527569c4d admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:58.784510 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 817} {{(pid=91643) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 20 04:21:58.806074 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:21:58.806553 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:58.808035 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] 10.222.0.74 "PUT /v2.0/ports/849868be-af51-4967-9a6c-93d23cecb450 HTTP/1.1" status: 200 len: 1094 time: 3.5577400 May 20 04:21:58.809034 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'e94a8b2b-8a29-45d4-9edb-63137be69ce4', 'tag': '849868be-af51-4967-9a6c-93d23cecb450'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:21:58.842937 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:58.936181 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Pushing event updated for resources: {'Port': ['ID=849868be-af51-4967-9a6c-93d23cecb450,revision_number=3']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:58.946529 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.258s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:59.000114 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=14b80be6-20fe-4dd3-860a-d08c679e3978,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:59.004978 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-202e1a79-0de0-4319-b075-d26e78b15aa7 req-3455f609-8175-4a1a-9333-c9e9bc2d2f77 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Synchronizing usage tracker for tenant:0b29d684ade544028fa17828cde46d56 on resource:security_group {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:21:59.033559 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.330s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:59.036146 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:59.036492 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.segments_db [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Added segment 78919ea5-5c05-448b-b3b0-4f555f1f9b88 of type vlan for network 2d8fc66e-0e6e-4447-b517-3622e5ccc480 May 20 04:21:59.045528 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f59e3df7-daf1-48c8-8662-386527569c4d admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:59.199975 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 68d41f86-efee-45dc-988a-b22c7179ec88 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:21:59.239518 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8760688516720'] for network, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:59.264263 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.db.resource_extend [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] It took 0.25 seconds to run function 'neutron_lib.db.resource_extend.apply_funcs' {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py:386}} May 20 04:21:59.374363 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.db.resource_extend [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] It took 0.12 seconds to run function 'neutron_lib.db.resource_extend.apply_funcs' {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py:386}} May 20 04:21:59.398074 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-202e1a79-0de0-4319-b075-d26e78b15aa7 req-3455f609-8175-4a1a-9333-c9e9bc2d2f77 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Unset dirty status for tenant:0b29d684ade544028fa17828cde46d56 on resource:security_group {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:21:59.399342 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-202e1a79-0de0-4319-b075-d26e78b15aa7 req-3455f609-8175-4a1a-9333-c9e9bc2d2f77 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.7291458 May 20 04:21:59.406202 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Notify callbacks [] for port, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:59.432587 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:59.435640 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 68d41f86-efee-45dc-988a-b22c7179ec88 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:21:59.436012 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:59.458727 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Allocated IP 172.27.0.153 (712ce8c4-8feb-4719-bf13-e06992125cd9/e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5/645c120d-b75d-4b37-a533-df616457b087) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:21:59.586829 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'e94a8b2b-8a29-45d4-9edb-63137be69ce4', 'tag': '849868be-af51-4967-9a6c-93d23cecb450', 'status': 'completed', 'code': 200} May 20 04:21:59.620543 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-7924897', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-8003163'] for router_interface, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:59.632568 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:21:59.669648 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915'] for network, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:59.692020 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:21:59.692616 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:59.907031 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f59e3df7-daf1-48c8-8662-386527569c4d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:21:59.938868 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Pushing event updated for resources: {'Port': ['ID=29b05a85-1060-41b1-aff1-48c1783a174f,revision_number=2']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:21:59.983516 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.351s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:21:59.997877 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.292s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:00.023242 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f59e3df7-daf1-48c8-8662-386527569c4d admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:00.024284 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-f59e3df7-daf1-48c8-8662-386527569c4d admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/931085ef-a427-4564-9d2e-443d5a6a21a9 HTTP/1.1" status: 204 len: 168 time: 1.7676470 May 20 04:22:00.056831 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] 10.222.0.39 "POST /v2.0/networks HTTP/1.1" status: 201 len: 898 time: 3.9488440 May 20 04:22:00.059943 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 33078) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:00.067227 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:00.074365 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-d8ba64c6-5cb5-4c5d-8c7b-d234f1cbb336 admin admin] Request body: {'subnet': {'network_id': '2d8fc66e-0e6e-4447-b517-3622e5ccc480', 'cidr': '172.20.0.0/28', 'name': 'tempest-EncryptedVolumesExtendAttachedTest-2048268485-subnet', 'tenant_id': '610f82a3f8c9479b83711aad8d2a1ca5', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:22:00.086834 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-d8ba64c6-5cb5-4c5d-8c7b-d234f1cbb336 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:22:00.096237 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-d8ba64c6-5cb5-4c5d-8c7b-d234f1cbb336 admin admin] Usage tracker for resource:subnet and tenant:610f82a3f8c9479b83711aad8d2a1ca5 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:22:00.105634 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-d8ba64c6-5cb5-4c5d-8c7b-d234f1cbb336 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:22:00.125347 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-d8ba64c6-5cb5-4c5d-8c7b-d234f1cbb336 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:22:00.168414 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-d8ba64c6-5cb5-4c5d-8c7b-d234f1cbb336 admin admin] Made reservation on behalf of 610f82a3f8c9479b83711aad8d2a1ca5 for: {'subnet': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:22:00.173703 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d8ba64c6-5cb5-4c5d-8c7b-d234f1cbb336 admin admin] Notify callbacks [] for subnet, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:00.174617 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Pushing event updated for resources: {'Network': ['ID=2d8fc66e-0e6e-4447-b517-3622e5ccc480,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:00.185072 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.187s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:00.187870 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.205s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:00.191453 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Network 2d8fc66e-0e6e-4447-b517-3622e5ccc480 details requested by agent ovs-agent-dv-cinder-782405-9-iscsi with host dv-cinder-782405-9-iscsi {{(pid=91645) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 20 04:22:00.204796 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Network 36b37213-8439-447d-9eba-a78baf7c698e is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:22:00.208745 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f59e3df7-daf1-48c8-8662-386527569c4d admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=931085ef-a427-4564-9d2e-443d5a6a21a9,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:00.210662 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:22:00.214142 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-f59e3df7-daf1-48c8-8662-386527569c4d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:00.326577 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-35dfed3e-5a64-46d2-a540-aa5ca204047c req-d679a680-2f01-4041-b8ac-1501d52de872 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:00.328139 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-35dfed3e-5a64-46d2-a540-aa5ca204047c req-d679a680-2f01-4041-b8ac-1501d52de872 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 944 time: 0.4172809 May 20 04:22:00.332196 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:00.343139 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:00.573458 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:22:00.581828 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:22:00.592658 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:22:00.604765 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Pushing event updated for resources: {'Port': ['ID=645c120d-b75d-4b37-a533-df616457b087,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:00.620577 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.277s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:00.632928 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-35dfed3e-5a64-46d2-a540-aa5ca204047c req-f9c69c3b-d596-4bd5-861f-f0ef5d67dfde tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2910047 May 20 04:22:00.748919 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Router be02e160-6134-4b60-b833-5a659cc82c44 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91643) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:22:00.767380 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for floatingip, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:00.901749 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d8ba64c6-5cb5-4c5d-8c7b-d234f1cbb336 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868'] for subnet, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:00.906607 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:00.940727 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-250ae839-1e77-458b-81a7-5fbd7e01f0ae req-bb621a90-318e-47a8-aada-c8e63fc17f26 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:00.950436 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-250ae839-1e77-458b-81a7-5fbd7e01f0ae req-bb621a90-318e-47a8-aada-c8e63fc17f26 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 931 time: 0.2766006 May 20 04:22:00.991284 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d8ba64c6-5cb5-4c5d-8c7b-d234f1cbb336 admin admin] Pushing event updated for resources: {'Subnet': ['ID=0429b3e4-01a6-4738-b92a-9810f3ecb32d,revision_number=0']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:00.995101 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d8ba64c6-5cb5-4c5d-8c7b-d234f1cbb336 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.088s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:01.003996 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-d8ba64c6-5cb5-4c5d-8c7b-d234f1cbb336 admin admin] Checking segment: {'id': '78919ea5-5c05-448b-b3b0-4f555f1f9b88', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 817, 'network_id': '2d8fc66e-0e6e-4447-b517-3622e5ccc480'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:22:01.004659 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d8ba64c6-5cb5-4c5d-8c7b-d234f1cbb336 admin admin] Network hostable DHCP agents. Network: 2d8fc66e-0e6e-4447-b517-3622e5ccc480, hostable agents: ['fffae25a-f450-4182-8f9e-ac29d143f6c9'], hosted agents: [] {{(pid=91644) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 20 04:22:01.015299 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:01.029228 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d8ba64c6-5cb5-4c5d-8c7b-d234f1cbb336 admin admin] Network 2d8fc66e-0e6e-4447-b517-3622e5ccc480 is scheduled to be hosted by DHCP agent fffae25a-f450-4182-8f9e-ac29d143f6c9 with binding_index 1 {{(pid=91644) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 20 04:22:01.046578 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-d8ba64c6-5cb5-4c5d-8c7b-d234f1cbb336 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-782405-9-iscsi, 2d8fc66e-0e6e-4447-b517-3622e5ccc480) {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:22:01.058395 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-02984b22-7236-4cd2-8afc-9b7522c3c28c None None] Network 2d8fc66e-0e6e-4447-b517-3622e5ccc480 requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:22:01.078563 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-d8ba64c6-5cb5-4c5d-8c7b-d234f1cbb336 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:01.079551 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d8ba64c6-5cb5-4c5d-8c7b-d234f1cbb336 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:01.121584 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-d8ba64c6-5cb5-4c5d-8c7b-d234f1cbb336 admin admin] 10.222.0.39 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 862 time: 1.0602634 May 20 04:22:01.145772 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-250ae839-1e77-458b-81a7-5fbd7e01f0ae req-901704f2-86ce-4f5c-a61f-c5868941b3f7 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1821527 May 20 04:22:01.152626 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:22:01.207266 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] 10.222.0.39 "PUT /v2.0/routers/be02e160-6134-4b60-b833-5a659cc82c44/add_router_interface HTTP/1.1" status: 200 len: 503 time: 10.4338822 May 20 04:22:01.207459 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Sync routers for ids ['be02e160-6134-4b60-b833-5a659cc82c44'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:22:01.232730 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Routers {'be02e160-6134-4b60-b833-5a659cc82c44'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:22:01.233479 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:21:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['be02e160-6134-4b60-b833-5a659cc82c44'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:22:01.356299 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for port, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:01.363584 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:01.369268 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-41679fab-3b54-466a-8bd4-6195bd67f2fe admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:01.454950 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-8003251'] for floatingip, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:01.474807 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e64f8269-a981-4362-9f97-d728e88deb95 None None] Network 2d8fc66e-0e6e-4447-b517-3622e5ccc480 requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:22:01.479806 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Pushing event updated for resources: {'Port': ['ID=645c120d-b75d-4b37-a533-df616457b087,revision_number=2']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:01.487326 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.116s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:01.491972 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:01.503008 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 693 time: 4.5944347 May 20 04:22:01.645473 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Set router be02e160-6134-4b60-b833-5a659cc82c44 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:22:01.962176 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e381fba9-4a97-4cbc-b1ac-cc7c880a9195 req-2b7054a0-fe65-4c65-88c2-18c24a90778b tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:01.974624 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e381fba9-4a97-4cbc-b1ac-cc7c880a9195 req-2b7054a0-fe65-4c65-88c2-18c24a90778b tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 944 time: 0.1954083 May 20 04:22:02.061855 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e381fba9-4a97-4cbc-b1ac-cc7c880a9195 req-81e84a20-4c11-4a4a-ad6b-0c3cfa0290cd tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0854948 May 20 04:22:02.295423 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'be02e160-6134-4b60-b833-5a659cc82c44', 'name': 'tempest-ServerRescueNegativeTestJSON-1026520542-router', 'tenant_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.120'}], 'enable_snat': True}, 'gw_port_id': '68d41f86-efee-45dc-988a-b22c7179ec88', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:43Z', 'updated_at': '2021-05-20T04:21:57Z', 'revision_number': 4, 'project_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'gw_port': {'id': '68d41f86-efee-45dc-988a-b22c7179ec88', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:65:dc:1e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'be02e160-6134-4b60-b833-5a659cc82c44', 'device_owner': 'network:router_gateway', 'standard_attr_id': 83, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.120', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:46Z', 'updated_at': '2021-05-20T04:21:57Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '29b05a85-1060-41b1-aff1-48c1783a174f', 'name': '', 'network_id': '36b37213-8439-447d-9eba-a78baf7c698e', 'tenant_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'mac_address': 'fa:16:3e:87:cd:92', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'be02e160-6134-4b60-b833-5a659cc82c44', 'device_owner': 'network:router_interface', 'standard_attr_id': 98, 'fixed_ips': [{'subnet_id': 'b25ae9c8-e11d-45ee-bde2-332cc1538602', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:53Z', 'updated_at': '2021-05-20T04:21:58Z', 'revision_number': 2, 'project_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'subnets': [{'id': 'b25ae9c8-e11d-45ee-bde2-332cc1538602', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:22:02.301186 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Checking router: be02e160-6134-4b60-b833-5a659cc82c44 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:22:02.301797 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Updating router be02e160-6134-4b60-b833-5a659cc82c44 port 29b05a85-1060-41b1-aff1-48c1783a174f binding host dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 20 04:22:02.386100 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:02.526708 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9da6316b-ea6c-44d1-9c2b-28483691c9df req-0fbd7f6e-4d26-49bd-b947-253e0235e723 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:02.529388 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9da6316b-ea6c-44d1-9c2b-28483691c9df req-0fbd7f6e-4d26-49bd-b947-253e0235e723 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 931 time: 0.2054434 May 20 04:22:02.800702 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:22:02.845227 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9da6316b-ea6c-44d1-9c2b-28483691c9df req-82e964a3-cb01-424e-9240-56c8170d7651 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.3034978 May 20 04:22:03.007391 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Notify callbacks [] for port, precommit_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:03.022987 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Transition to ACTIVE for port object 29b05a85-1060-41b1-aff1-48c1783a174f will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:22:03.036053 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:03.043301 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:03.048309 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 33208) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:03.054831 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Request body: {'network': {'name': 'tempest-EncryptedVolumesExtendAttachedTest-511357591-network', 'tenant_id': '5d94f8f158c1463ab47e1dfd2ae612a3'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:22:03.063882 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:22:03.084420 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Usage tracker for resource:network and tenant:5d94f8f158c1463ab47e1dfd2ae612a3 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:22:03.093080 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:22:03.108519 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:22:03.126277 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Made reservation on behalf of 5d94f8f158c1463ab47e1dfd2ae612a3 for: {'network': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:22:03.134716 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-585852'] for network, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:03.145273 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1aeae814-b2cf-41fb-8dec-30c2d92c73b8 None None] Pushing event updated for resources: {'Port': ['ID=29b05a85-1060-41b1-aff1-48c1783a174f,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:03.159312 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-1aeae814-b2cf-41fb-8dec-30c2d92c73b8 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.116s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:03.163069 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-034f1eee-144b-49ca-af9a-0682bc622247 req-50ba9b79-c8cf-482b-b404-6b4eaf10eefc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?name=tempest-securitygroup--635106175&fields=id&tenant_id=9fafdf3a29024ac588e5b24719735677 HTTP/1.1" status: 200 len: 267 time: 0.0746136 May 20 04:22:03.172697 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:03.241058 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:22:03.252233 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Usage tracker for resource:security_group_rule and tenant:5d94f8f158c1463ab47e1dfd2ae612a3 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:22:03.264335 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:22:03.282418 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:22:03.310820 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-09d4009c-634d-4879-8636-d1183aee63c4 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:03.318199 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-09d4009c-634d-4879-8636-d1183aee63c4 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Synchronizing usage tracker for tenant:bec56d8703cf47f3b7193b7d1f55396d on resource:network {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:22:03.504029 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:03.513136 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:03.514619 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Attempting to bind port 29b05a85-1060-41b1-aff1-48c1783a174f on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:22:03.515342 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Attempting to bind port 29b05a85-1060-41b1-aff1-48c1783a174f by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '0032d9a3-07bf-489f-bb9d-03ab3ff14f98', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 813, 'network_id': '36b37213-8439-447d-9eba-a78baf7c698e'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:22:03.516454 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Attempting to bind port 29b05a85-1060-41b1-aff1-48c1783a174f on network 36b37213-8439-447d-9eba-a78baf7c698e {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:22:03.527319 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 21, 45), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:22:03.527319 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Checking segment: {'id': '0032d9a3-07bf-489f-bb9d-03ab3ff14f98', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 813, 'network_id': '36b37213-8439-447d-9eba-a78baf7c698e'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:22:03.527319 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:03.527319 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Bound using segment: {'id': '0032d9a3-07bf-489f-bb9d-03ab3ff14f98', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 813, 'network_id': '36b37213-8439-447d-9eba-a78baf7c698e'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:22:03.527319 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Bound port: 29b05a85-1060-41b1-aff1-48c1783a174f, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '0032d9a3-07bf-489f-bb9d-03ab3ff14f98', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 813, 'network_id': '36b37213-8439-447d-9eba-a78baf7c698e'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:22:03.527319 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:03.546920 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 823} {{(pid=91644) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 20 04:22:03.578860 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:03.579127 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.db.segments_db [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Added segment d7c9e7f1-8fb6-4935-bdee-61056fc295ff of type vlan for network 3455067b-ac51-440f-a307-32c101262582 May 20 04:22:03.636992 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-09d4009c-634d-4879-8636-d1183aee63c4 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Unset dirty status for tenant:bec56d8703cf47f3b7193b7d1f55396d on resource:network {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:22:03.638413 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-09d4009c-634d-4879-8636-d1183aee63c4 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/networks?id=36b37213-8439-447d-9eba-a78baf7c698e HTTP/1.1" status: 200 len: 819 time: 0.6894462 May 20 04:22:03.655522 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] For port 29b05a85-1060-41b1-aff1-48c1783a174f, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:22:03.659239 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] For port 29b05a85-1060-41b1-aff1-48c1783a174f, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=29b05a85-1060-41b1-aff1-48c1783a174f,segment=NetworkSegment(0032d9a3-07bf-489f-bb9d-03ab3ff14f98),segment_id=0032d9a3-07bf-489f-bb9d-03ab3ff14f98)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:22:03.666219 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:03.668390 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-71933fe8-dfea-47b9-b610-dcfe642e1cc4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:03.678121 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-1aeae814-b2cf-41fb-8dec-30c2d92c73b8 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:03.701018 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8760688516720'] for network, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:03.723383 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-0042c721-aaf2-45c0-8e52-865339919133 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/quotas/bec56d8703cf47f3b7193b7d1f55396d HTTP/1.1" status: 200 len: 375 time: 0.0592554 May 20 04:22:03.756645 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=093f0751-ee16-49cc-b807-ef912b257abc,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:03.762635 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.094s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:03.772996 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Routers {'be02e160-6134-4b60-b833-5a659cc82c44'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:22:03.773707 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:21:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['be02e160-6134-4b60-b833-5a659cc82c44'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:22:03.823063 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915'] for network, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:03.823063 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:03.823776 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:03.858321 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:03.905854 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-034f1eee-144b-49ca-af9a-0682bc622247 req-13103cd1-a315-488c-ba91-00d89d5e64da tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:03.908948 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-034f1eee-144b-49ca-af9a-0682bc622247 req-13103cd1-a315-488c-ba91-00d89d5e64da tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/networks?id=64bcea8a-6d35-406b-a823-fa1185008174 HTTP/1.1" status: 200 len: 817 time: 0.7167773 May 20 04:22:03.975292 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-034f1eee-144b-49ca-af9a-0682bc622247 req-756676c7-84e4-4cb4-be1f-f432baa5eb7d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/quotas/9fafdf3a29024ac588e5b24719735677 HTTP/1.1" status: 200 len: 375 time: 0.0320792 May 20 04:22:03.979797 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-8b5d510c-080c-4cbe-bdbc-82c54991be48 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Synchronizing usage tracker for tenant:bec56d8703cf47f3b7193b7d1f55396d on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:22:04.037687 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] Set router be02e160-6134-4b60-b833-5a659cc82c44 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:22:04.046721 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Pushing event updated for resources: {'Network': ['ID=3455067b-ac51-440f-a307-32c101262582,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:04.065782 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] 10.222.0.39 "POST /v2.0/networks HTTP/1.1" status: 201 len: 897 time: 1.0157974 May 20 04:22:04.067788 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.210s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:04.073349 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 33254) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:04.086528 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-c38d0570-3d6e-4f02-95a0-aa6bea59d014 admin admin] Request body: {'subnet': {'network_id': '3455067b-ac51-440f-a307-32c101262582', 'cidr': '172.20.0.0/28', 'name': 'tempest-EncryptedVolumesExtendAttachedTest-511357591-subnet', 'tenant_id': '5d94f8f158c1463ab47e1dfd2ae612a3', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:22:04.104185 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-c38d0570-3d6e-4f02-95a0-aa6bea59d014 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:22:04.129515 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-c38d0570-3d6e-4f02-95a0-aa6bea59d014 admin admin] Usage tracker for resource:subnet and tenant:5d94f8f158c1463ab47e1dfd2ae612a3 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:22:04.140493 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-c38d0570-3d6e-4f02-95a0-aa6bea59d014 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:22:04.142352 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-8b5d510c-080c-4cbe-bdbc-82c54991be48 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Unset dirty status for tenant:bec56d8703cf47f3b7193b7d1f55396d on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:22:04.143869 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-8b5d510c-080c-4cbe-bdbc-82c54991be48 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=bec56d8703cf47f3b7193b7d1f55396d&fields=id HTTP/1.1" status: 200 len: 255 time: 0.4085069 May 20 04:22:04.159455 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-c38d0570-3d6e-4f02-95a0-aa6bea59d014 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:22:04.214587 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-c38d0570-3d6e-4f02-95a0-aa6bea59d014 admin admin] Made reservation on behalf of 5d94f8f158c1463ab47e1dfd2ae612a3 for: {'subnet': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:22:04.218821 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c38d0570-3d6e-4f02-95a0-aa6bea59d014 admin admin] Notify callbacks [] for subnet, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:04.235036 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Network 3455067b-ac51-440f-a307-32c101262582 details requested by agent ovs-agent-dv-cinder-782405-9-iscsi with host dv-cinder-782405-9-iscsi {{(pid=91645) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 20 04:22:04.278237 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-034f1eee-144b-49ca-af9a-0682bc622247 req-ed5cca98-0223-4411-b9f3-dbbc448af2ab tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=9fafdf3a29024ac588e5b24719735677&fields=id HTTP/1.1" status: 200 len: 348 time: 0.2885354 May 20 04:22:04.520716 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1aeae814-b2cf-41fb-8dec-30c2d92c73b8 None None] Pushing event updated for resources: {'Port': ['ID=29b05a85-1060-41b1-aff1-48c1783a174f,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:04.524379 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-1aeae814-b2cf-41fb-8dec-30c2d92c73b8 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.846s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:04.610708 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-3a00ee34-75a1-4ac2-9952-313f8fd01450 service neutron] 10.222.0.39 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.4577768 May 20 04:22:04.632553 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'be02e160-6134-4b60-b833-5a659cc82c44', 'name': 'tempest-ServerRescueNegativeTestJSON-1026520542-router', 'tenant_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.120'}], 'enable_snat': True}, 'gw_port_id': '68d41f86-efee-45dc-988a-b22c7179ec88', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:43Z', 'updated_at': '2021-05-20T04:21:57Z', 'revision_number': 4, 'project_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'gw_port': {'id': '68d41f86-efee-45dc-988a-b22c7179ec88', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:65:dc:1e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'be02e160-6134-4b60-b833-5a659cc82c44', 'device_owner': 'network:router_gateway', 'standard_attr_id': 83, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.120', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:46Z', 'updated_at': '2021-05-20T04:21:57Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '29b05a85-1060-41b1-aff1-48c1783a174f', 'name': '', 'network_id': '36b37213-8439-447d-9eba-a78baf7c698e', 'tenant_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'mac_address': 'fa:16:3e:87:cd:92', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'be02e160-6134-4b60-b833-5a659cc82c44', 'device_owner': 'network:router_interface', 'standard_attr_id': 98, 'fixed_ips': [{'subnet_id': 'b25ae9c8-e11d-45ee-bde2-332cc1538602', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:53Z', 'updated_at': '2021-05-20T04:22:03Z', 'revision_number': 4, 'project_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'subnets': [{'id': 'b25ae9c8-e11d-45ee-bde2-332cc1538602', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:22:04.637479 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-859ef9ab-6be2-4372-a79a-2063fc1a8e8e None None] The sync data for ids ['be02e160-6134-4b60-b833-5a659cc82c44'] in dv-cinder-782405-9-iscsi is: [{'id': 'be02e160-6134-4b60-b833-5a659cc82c44', 'name': 'tempest-ServerRescueNegativeTestJSON-1026520542-router', 'tenant_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.120'}], 'enable_snat': True}, 'gw_port_id': '68d41f86-efee-45dc-988a-b22c7179ec88', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:43Z', 'updated_at': '2021-05-20T04:21:57Z', 'revision_number': 4, 'project_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'gw_port': {'id': '68d41f86-efee-45dc-988a-b22c7179ec88', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:65:dc:1e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'be02e160-6134-4b60-b833-5a659cc82c44', 'device_owner': 'network:router_gateway', 'standard_attr_id': 83, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.120', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:46Z', 'updated_at': '2021-05-20T04:21:57Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '29b05a85-1060-41b1-aff1-48c1783a174f', 'name': '', 'network_id': '36b37213-8439-447d-9eba-a78baf7c698e', 'tenant_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'mac_address': 'fa:16:3e:87:cd:92', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'be02e160-6134-4b60-b833-5a659cc82c44', 'device_owner': 'network:router_interface', 'standard_attr_id': 98, 'fixed_ips': [{'subnet_id': 'b25ae9c8-e11d-45ee-bde2-332cc1538602', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:53Z', 'updated_at': '2021-05-20T04:22:03Z', 'revision_number': 4, 'project_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'subnets': [{'id': 'b25ae9c8-e11d-45ee-bde2-332cc1538602', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:22:04.683266 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-034f1eee-144b-49ca-af9a-0682bc622247 req-f1ab4d5a-3d5e-4769-8dc8-51bce3da7641 service neutron] 10.222.0.39 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3085563 May 20 04:22:04.897328 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c38d0570-3d6e-4f02-95a0-aa6bea59d014 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868'] for subnet, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:04.901449 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:04.959847 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-034f1eee-144b-49ca-af9a-0682bc622247 req-c4063a57-e9df-4a4d-a491-a32b38e30490 service neutron] 10.222.0.39 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2090266 May 20 04:22:04.970582 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-aea59bef-2123-41bb-b079-0568bc2775de service neutron] 10.222.0.39 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3496783 May 20 04:22:05.008049 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c38d0570-3d6e-4f02-95a0-aa6bea59d014 admin admin] Pushing event updated for resources: {'Subnet': ['ID=7a7294d2-afd0-4235-a9fe-0b3c672f13bc,revision_number=0']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:05.011077 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c38d0570-3d6e-4f02-95a0-aa6bea59d014 admin admin] Checking segment: {'id': 'd7c9e7f1-8fb6-4935-bdee-61056fc295ff', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 823, 'network_id': '3455067b-ac51-440f-a307-32c101262582'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:22:05.011915 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c38d0570-3d6e-4f02-95a0-aa6bea59d014 admin admin] Network hostable DHCP agents. Network: 3455067b-ac51-440f-a307-32c101262582, hostable agents: ['fffae25a-f450-4182-8f9e-ac29d143f6c9'], hosted agents: [] {{(pid=91644) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 20 04:22:05.021092 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-c38d0570-3d6e-4f02-95a0-aa6bea59d014 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.120s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:05.050686 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c38d0570-3d6e-4f02-95a0-aa6bea59d014 admin admin] Network 3455067b-ac51-440f-a307-32c101262582 is scheduled to be hosted by DHCP agent fffae25a-f450-4182-8f9e-ac29d143f6c9 with binding_index 1 {{(pid=91644) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 20 04:22:05.067308 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-c38d0570-3d6e-4f02-95a0-aa6bea59d014 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-782405-9-iscsi, 3455067b-ac51-440f-a307-32c101262582) {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:22:05.078518 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-857ff9cf-d021-4a6a-a59a-a03a69e8ae6e None None] Network 3455067b-ac51-440f-a307-32c101262582 requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:22:05.092635 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-c38d0570-3d6e-4f02-95a0-aa6bea59d014 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:05.093330 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c38d0570-3d6e-4f02-95a0-aa6bea59d014 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:05.178055 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-c38d0570-3d6e-4f02-95a0-aa6bea59d014 admin admin] 10.222.0.39 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 861 time: 1.0980973 May 20 04:22:05.397351 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 29b05a85-1060-41b1-aff1-48c1783a174f up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:22:05.633186 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 33316) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:05.755818 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:05.808225 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_v2 [None req-31ca805b-6bf6-4f9c-b1a9-171a1016b2a8 admin admin] Deleting subnet 0429b3e4-01a6-4738-b92a-9810f3ecb32d {{(pid=91644) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 20 04:22:05.815888 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-31ca805b-6bf6-4f9c-b1a9-171a1016b2a8 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:05.930163 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c5e011f6-4673-4dea-ad83-8f598eb35f5a req-4dbe13f9-e7ae-45fa-92ea-09389acb5e2d tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:05.956360 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c5e011f6-4673-4dea-ad83-8f598eb35f5a req-4dbe13f9-e7ae-45fa-92ea-09389acb5e2d tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 944 time: 0.5098567 May 20 04:22:05.989585 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-31ca805b-6bf6-4f9c-b1a9-171a1016b2a8 admin admin] Notify callbacks [] for subnet, precommit_delete_associations {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:06.054182 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-31ca805b-6bf6-4f9c-b1a9-171a1016b2a8 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-7786687'] for subnet, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:06.094951 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port 29b05a85-1060-41b1-aff1-48c1783a174f completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:22:06.106865 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c5e011f6-4673-4dea-ad83-8f598eb35f5a req-2ff1f358-5339-44b2-989c-17f18d7c37a7 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1494877 May 20 04:22:06.112084 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-ec7bc68a-d94c-43e9-ae43-327b15831193 None None] Network 3455067b-ac51-440f-a307-32c101262582 requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:22:06.134023 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-af870063-4615-4ab8-a89f-61ea6b6939a7 req-50a86e5a-afd1-4f2a-981d-8239cd3f9dad tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:06.153985 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-af870063-4615-4ab8-a89f-61ea6b6939a7 req-50a86e5a-afd1-4f2a-981d-8239cd3f9dad tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 931 time: 0.4210181 May 20 04:22:06.374345 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-af870063-4615-4ab8-a89f-61ea6b6939a7 req-3414b387-47ec-467a-9c34-50b3d4a5f735 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2184522 May 20 04:22:06.375661 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Request body: {'port': {'device_id': 'bea4be17-559d-407e-9b61-ff2cb94317dc', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:22:06.421614 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9d32c5d0-3f0a-40e4-94e6-8e436ba0cedf req-2e38bdde-a8c4-48e6-8bed-d16d60e8f8c8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 210 time: 0.1649830 May 20 04:22:06.457958 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 33338) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:06.519604 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 29b05a85-1060-41b1-aff1-48c1783a174f triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:22:06.519997 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:06.569170 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-af3dd9e2-60e9-46de-8f71-52420e93776e req-29b709c4-e2fd-40b0-81a4-a2d2c0988fe5 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 210 time: 0.0965974 May 20 04:22:06.694652 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c16dac2a-83e9-447f-8b47-8c285f12960c admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:06.712200 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-c16dac2a-83e9-447f-8b47-8c285f12960c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:22:06.717671 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-c16dac2a-83e9-447f-8b47-8c285f12960c admin admin] Usage tracker for resource:security_group_rule and tenant:e2d4bec9becf4a4292fd57391f8ec105 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:22:06.722066 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-c16dac2a-83e9-447f-8b47-8c285f12960c admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:22:06.735132 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-c16dac2a-83e9-447f-8b47-8c285f12960c admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:22:06.863183 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c16dac2a-83e9-447f-8b47-8c285f12960c admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:06.873665 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c16dac2a-83e9-447f-8b47-8c285f12960c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:06.917446 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:06.948834 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-c16dac2a-83e9-447f-8b47-8c285f12960c admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=e2d4bec9becf4a4292fd57391f8ec105&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4896996 May 20 04:22:06.951249 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c16dac2a-83e9-447f-8b47-8c285f12960c admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=2adebaf9-d5e6-4006-b596-e49dcd276ff6,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:06.954233 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 33364) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:06.958000 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-c16dac2a-83e9-447f-8b47-8c285f12960c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.040s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:06.985700 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:07.026814 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:07.030189 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port 29b05a85-1060-41b1-aff1-48c1783a174f, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=29b05a85-1060-41b1-aff1-48c1783a174f,segment=NetworkSegment(0032d9a3-07bf-489f-bb9d-03ab3ff14f98),segment_id=0032d9a3-07bf-489f-bb9d-03ab3ff14f98)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:22:07.035396 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:07.042895 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:07.075625 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-daad56e3-7e95-4095-b8cb-d916b6ea90f4 req-14e017f1-8eef-4869-b694-049d3a659efd service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:07.085144 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-daad56e3-7e95-4095-b8cb-d916b6ea90f4 req-14e017f1-8eef-4869-b694-049d3a659efd service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=51fcbc78284541db832e4780e76158af&device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 1097 time: 0.5830040 May 20 04:22:07.155167 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=29b05a85-1060-41b1-aff1-48c1783a174f,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:07.179627 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.134s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:07.441579 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-ff4d28c8-7a6a-4541-a826-db277f98cbe1 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:07.443415 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-ff4d28c8-7a6a-4541-a826-db277f98cbe1 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/networks?id=36b37213-8439-447d-9eba-a78baf7c698e HTTP/1.1" status: 200 len: 819 time: 0.4933941 May 20 04:22:07.475054 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-48e57dc6-e1e3-4200-a9d8-67faeacdb318 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:07.537236 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-31ca805b-6bf6-4f9c-b1a9-171a1016b2a8 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-7786632', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868'] for subnet, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:07.568324 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:07.570552 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-6e1e1325-ae57-4fb1-98ff-2098abff5c48 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/quotas/bec56d8703cf47f3b7193b7d1f55396d HTTP/1.1" status: 200 len: 375 time: 0.1002228 May 20 04:22:07.609841 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-31ca805b-6bf6-4f9c-b1a9-171a1016b2a8 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=0429b3e4-01a6-4738-b92a-9810f3ecb32d,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:07.611781 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-31ca805b-6bf6-4f9c-b1a9-171a1016b2a8 admin admin] Notify callbacks [] for subnet, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:07.622135 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-31ca805b-6bf6-4f9c-b1a9-171a1016b2a8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:07.624998 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-48e57dc6-e1e3-4200-a9d8-67faeacdb318 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:07.639227 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-daad56e3-7e95-4095-b8cb-d916b6ea90f4 req-89eaa43d-702b-45f3-bf44-76e7177fbdd7 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:07.640731 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-daad56e3-7e95-4095-b8cb-d916b6ea90f4 req-89eaa43d-702b-45f3-bf44-76e7177fbdd7 service neutron] 10.222.0.74 "GET /v2.0/networks?tenant_id=51fcbc78284541db832e4780e76158af&shared=False HTTP/1.1" status: 200 len: 916 time: 0.5355194 May 20 04:22:07.649971 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e680746e-985c-4f4f-952b-1cd9abcebcfb req-e5607b38-21a4-4619-98b6-5bbc84540b76 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:07.651717 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e680746e-985c-4f4f-952b-1cd9abcebcfb req-e5607b38-21a4-4619-98b6-5bbc84540b76 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 944 time: 0.3479900 May 20 04:22:07.670771 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-31ca805b-6bf6-4f9c-b1a9-171a1016b2a8 admin admin] 10.222.0.39 "DELETE /v2.0/subnets/0429b3e4-01a6-4738-b92a-9810f3ecb32d HTTP/1.1" status: 204 len: 168 time: 2.0340796 May 20 04:22:07.671589 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 33392) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:07.742791 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e680746e-985c-4f4f-952b-1cd9abcebcfb req-23ef94d9-aa62-4a3a-999f-47ee08178704 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0787747 May 20 04:22:07.758967 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-b9e876a0-aed8-48a3-b6ea-0cb575324b6e tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=bec56d8703cf47f3b7193b7d1f55396d&fields=id HTTP/1.1" status: 200 len: 255 time: 0.1771674 May 20 04:22:07.769387 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-48e57dc6-e1e3-4200-a9d8-67faeacdb318 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:07.774063 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:07.788306 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.74', 56953) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:07.843038 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-48e57dc6-e1e3-4200-a9d8-67faeacdb318 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=2adebaf9-d5e6-4006-b596-e49dcd276ff6,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:07.870372 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-48e57dc6-e1e3-4200-a9d8-67faeacdb318 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:07.906874 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-48e57dc6-e1e3-4200-a9d8-67faeacdb318 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:07.910319 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-48e57dc6-e1e3-4200-a9d8-67faeacdb318 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/2adebaf9-d5e6-4006-b596-e49dcd276ff6 HTTP/1.1" status: 204 len: 168 time: 0.9518905 May 20 04:22:08.014642 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-daad56e3-7e95-4095-b8cb-d916b6ea90f4 req-67a23ffe-539c-45f3-96ce-05ad6061981e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:08.016165 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-daad56e3-7e95-4095-b8cb-d916b6ea90f4 req-67a23ffe-539c-45f3-96ce-05ad6061981e service neutron] 10.222.0.74 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.2474098 May 20 04:22:08.029904 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-593607'] for network, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:08.177808 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-4be744a3-fda2-48e2-b7fe-53670874398d service neutron] 10.222.0.39 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3874269 May 20 04:22:08.185364 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-034f1eee-144b-49ca-af9a-0682bc622247 req-87bcc116-31a1-4746-91d8-82efdd9f749a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:08.186827 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-034f1eee-144b-49ca-af9a-0682bc622247 req-87bcc116-31a1-4746-91d8-82efdd9f749a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.74 "GET /v2.0/networks?id=64bcea8a-6d35-406b-a823-fa1185008174 HTTP/1.1" status: 200 len: 817 time: 0.3953276 May 20 04:22:08.233990 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-7789108', 'neutron.services.segments.db._delete_segments_for_network-8760688958082'] for network, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:08.237918 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bcecb104-7555-4bd8-a92a-d7552857bce9 req-3f75c554-0f9b-488c-b87a-9ce147c85c16 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:08.239426 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bcecb104-7555-4bd8-a92a-d7552857bce9 req-3f75c554-0f9b-488c-b87a-9ce147c85c16 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 931 time: 0.2755284 May 20 04:22:08.253558 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 33406) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:08.257153 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Subscribe: > segment before_delete 55550000 {{(pid=91644) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:22:08.257490 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Subscribe: > subnet precommit_delete_associations 55550000 {{(pid=91644) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:22:08.258297 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Subscribe: > segment_host_mapping after_create 55550000 {{(pid=91644) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:22:08.258635 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Subscribe: > port after_create 55550000 {{(pid=91644) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:22:08.258982 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Subscribe: > port after_delete 55550000 {{(pid=91644) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:22:08.259316 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Subscribe: > port after_update 55550000 {{(pid=91644) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:22:08.259630 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Subscribe: > segment after_delete 55550000 {{(pid=91644) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:22:08.259942 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Subscribe: > subnet after_create 55550000 {{(pid=91644) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:22:08.260249 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Subscribe: > subnet after_delete 55550000 {{(pid=91644) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:22:08.260579 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Subscribe: > subnet after_update 55550000 {{(pid=91644) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:22:08.348508 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bcecb104-7555-4bd8-a92a-d7552857bce9 req-ff446aa0-ecda-4f5b-a368-aa66793b7030 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0629430 May 20 04:22:08.357027 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-e5941001-7c9c-4e80-9b2a-8b8276c7cb6d service neutron] 10.222.0.39 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1691060 May 20 04:22:08.374872 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Subscribe: > subnet after_create 55550000 {{(pid=91644) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:22:08.375599 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Subscribe: > subnet after_delete 55550000 {{(pid=91644) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:22:08.375998 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Subscribe: > subnet before_create 55550000 {{(pid=91644) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:22:08.376387 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Subscribe: > subnet before_update 55550000 {{(pid=91644) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:22:08.376877 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['2d8fc66e-0e6e-4447-b517-3622e5ccc480']}} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:22:08.409045 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id efb81444-f373-4e9c-a0aa-e3270bd4b980 {{(pid=91644) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:22:08.414814 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '78919ea5-5c05-448b-b3b0-4f555f1f9b88') {'for_net_delete': True} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:22:08.415251 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '78919ea5-5c05-448b-b3b0-4f555f1f9b88') {} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:22:08.422011 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-12233e74-cb7e-48a5-881a-49d4928129bd admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:08.439352 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8760688968331', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-853278'] for segment, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:08.446696 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:08.464632 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:08.472275 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-12233e74-cb7e-48a5-881a-49d4928129bd admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:22:08.487779 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Releasing vlan 817 on physical network physnet1 to pool {{(pid=91644) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 20 04:22:08.488502 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2001055'] for segment, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:08.500751 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-329b5d26-b665-4272-88c2-284b28689bff req-a7a4773c-37d5-4cdd-bea6-158e35e58b62 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 210 time: 0.0507119 May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: from cryptography.utils import int_from_bytes May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: from cryptography.utils import int_from_bytes May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: Traceback (most recent call last): May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: timer() May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: cb(*args, **kw) May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: return func(*args, **kwargs) May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: self._notify() May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: self.callback(batched_events) May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: event.method(event) May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: aggregate_id = self._get_aggregate_id(event.segment_id) May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: aggregate_uuid = self.p_client.list_aggregates( May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: return f(self, *a, **k) May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: return self._get(url).json() May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: return self._client.get(url, endpoint_filter=self._ks_filter, May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: return self.request(url, 'GET', **kwargs) May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: base_url = self.get_endpoint(auth, allow=allow, May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: auth = self._auth_required(auth, 'determine endpoint URL') May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 20 04:22:08.519258 dv-cinder-782405-9-iscsi neutron-server[91644]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 20 04:22:08.536425 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-12233e74-cb7e-48a5-881a-49d4928129bd admin admin] Usage tracker for resource:security_group_rule and tenant:83fbf76e08b8489c8fc6c35fdd5b9be7 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:22:08.541856 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-12233e74-cb7e-48a5-881a-49d4928129bd admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:22:08.574765 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5c340dcb-c778-4cdd-bcf4-b7b596997257 req-2419ebd6-925f-4525-accc-26355214c8ae tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 210 time: 0.0576348 May 20 04:22:08.600513 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-12233e74-cb7e-48a5-881a-49d4928129bd admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:22:08.635778 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Transition to ACTIVE for port object efb81444-f373-4e9c-a0aa-e3270bd4b980 will not be triggered until provisioned by entity L2. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:22:08.638090 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-7789117', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-8053321'] for network, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:08.644867 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:08.655325 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-1f6a72f9-924e-4235-af89-79bf16a97548 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:08.695320 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:08.798782 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] 10.222.0.39 "DELETE /v2.0/networks/2d8fc66e-0e6e-4447-b517-3622e5ccc480 HTTP/1.1" status: 204 len: 168 time: 1.1264031 May 20 04:22:08.801713 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 33422) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:08.852323 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Pushing event updated for resources: {'Port': ['ID=efb81444-f373-4e9c-a0aa-e3270bd4b980,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:08.866912 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.212s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:08.885006 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_v2 [None req-a253eddc-9e40-4c7f-b691-30b352c841ac admin admin] Deleting subnet 7a7294d2-afd0-4235-a9fe-0b3c672f13bc {{(pid=91643) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 20 04:22:08.888488 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a253eddc-9e40-4c7f-b691-30b352c841ac admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:08.894276 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.211s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:08.925314 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Pushing event deleted for resources: {'Network': ['ID=2d8fc66e-0e6e-4447-b517-3622e5ccc480,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:08.940367 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-dc70bfab-9e15-489d-9026-f642c252d6dc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:08.957550 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-12233e74-cb7e-48a5-881a-49d4928129bd admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:08.977355 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Attempting to bind port efb81444-f373-4e9c-a0aa-e3270bd4b980 on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:22:08.977619 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Attempting to bind port efb81444-f373-4e9c-a0aa-e3270bd4b980 by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': '3c4a2238-8a32-4d92-a1b1-8af91c1c27a7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 814, 'network_id': 'a009b619-8ab3-40e3-b5f3-9cc97ccd3415'}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:22:08.977931 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Attempting to bind port efb81444-f373-4e9c-a0aa-e3270bd4b980 on network a009b619-8ab3-40e3-b5f3-9cc97ccd3415 {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:22:08.981556 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-12233e74-cb7e-48a5-881a-49d4928129bd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:08.989607 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Port efb81444-f373-4e9c-a0aa-e3270bd4b980 on network a009b619-8ab3-40e3-b5f3-9cc97ccd3415 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:22:08.989760 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Attempting to bind port efb81444-f373-4e9c-a0aa-e3270bd4b980 on network a009b619-8ab3-40e3-b5f3-9cc97ccd3415 {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:22:08.991575 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a253eddc-9e40-4c7f-b691-30b352c841ac admin admin] Notify callbacks [] for subnet, precommit_delete_associations {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:08.994290 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:09.002602 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a253eddc-9e40-4c7f-b691-30b352c841ac admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-7786687'] for subnet, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:09.026220 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-12233e74-cb7e-48a5-881a-49d4928129bd admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=0b1d85f7-0557-4d6c-b22a-e73686fa56e4,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:09.029034 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 22, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:22:09.029368 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Checking segment: {'id': '3c4a2238-8a32-4d92-a1b1-8af91c1c27a7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 814, 'network_id': 'a009b619-8ab3-40e3-b5f3-9cc97ccd3415'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:22:09.029742 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Bound using segment: {'id': '3c4a2238-8a32-4d92-a1b1-8af91c1c27a7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 814, 'network_id': 'a009b619-8ab3-40e3-b5f3-9cc97ccd3415'} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:22:09.030185 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Bound port: efb81444-f373-4e9c-a0aa-e3270bd4b980, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '3c4a2238-8a32-4d92-a1b1-8af91c1c27a7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 814, 'network_id': 'a009b619-8ab3-40e3-b5f3-9cc97ccd3415'}}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:22:09.030535 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:09.039682 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-12233e74-cb7e-48a5-881a-49d4928129bd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:09.089877 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-12233e74-cb7e-48a5-881a-49d4928129bd admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=83fbf76e08b8489c8fc6c35fdd5b9be7&name=default HTTP/1.1" status: 200 len: 2979 time: 0.8327610 May 20 04:22:09.091682 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 33432) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:09.101489 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-50fdee32-011c-4d80-b645-7bc126a87ae3 req-fbd85a83-9032-4a93-9a1d-707470aa2b82 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:09.103113 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-50fdee32-011c-4d80-b645-7bc126a87ae3 req-fbd85a83-9032-4a93-9a1d-707470aa2b82 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 944 time: 0.1878448 May 20 04:22:09.142420 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] For port efb81444-f373-4e9c-a0aa-e3270bd4b980, host hv-cinder-78240, cleared binding levels {{(pid=91644) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:22:09.144219 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 29b05a85-1060-41b1-aff1-48c1783a174f up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:22:09.146085 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] For port efb81444-f373-4e9c-a0aa-e3270bd4b980, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=efb81444-f373-4e9c-a0aa-e3270bd4b980,segment=NetworkSegment(3c4a2238-8a32-4d92-a1b1-8af91c1c27a7),segment_id=3c4a2238-8a32-4d92-a1b1-8af91c1c27a7)] {{(pid=91644) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:22:09.152917 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:09.159898 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-ec91e9d5-a4de-4741-a4c5-5872927e6cf4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:09.225217 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:09.225786 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 29b05a85-1060-41b1-aff1-48c1783a174f triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:22:09.225979 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:09.226621 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:09.227655 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] 10.222.0.74 "PUT /v2.0/ports/efb81444-f373-4e9c-a0aa-e3270bd4b980 HTTP/1.1" status: 200 len: 1093 time: 3.1870046 May 20 04:22:09.228178 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'bea4be17-559d-407e-9b61-ff2cb94317dc', 'tag': 'efb81444-f373-4e9c-a0aa-e3270bd4b980'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:22:09.242799 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Pushing event updated for resources: {'Port': ['ID=efb81444-f373-4e9c-a0aa-e3270bd4b980,revision_number=3']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:09.246636 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.087s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:09.259843 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-50fdee32-011c-4d80-b645-7bc126a87ae3 req-b2bf7995-d9b5-4bd0-84c7-4f8b7c65d7ab tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1441631 May 20 04:22:09.301199 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9fb3fca5-141d-4af5-82e9-55f49eeb933c admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:09.323042 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'bea4be17-559d-407e-9b61-ff2cb94317dc', 'tag': 'efb81444-f373-4e9c-a0aa-e3270bd4b980', 'status': 'completed', 'code': 200} May 20 04:22:09.371815 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9fb3fca5-141d-4af5-82e9-55f49eeb933c admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:09.424403 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a253eddc-9e40-4c7f-b691-30b352c841ac admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-7786632', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868'] for subnet, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:09.428799 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:09.466689 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a253eddc-9e40-4c7f-b691-30b352c841ac admin admin] Pushing event deleted for resources: {'Subnet': ['ID=7a7294d2-afd0-4235-a9fe-0b3c672f13bc,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:09.470073 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a253eddc-9e40-4c7f-b691-30b352c841ac admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:09.509628 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-a253eddc-9e40-4c7f-b691-30b352c841ac admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:09.510235 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9fb3fca5-141d-4af5-82e9-55f49eeb933c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:09.520236 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9fb3fca5-141d-4af5-82e9-55f49eeb933c admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:09.521302 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-9fb3fca5-141d-4af5-82e9-55f49eeb933c admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/0b1d85f7-0557-4d6c-b22a-e73686fa56e4 HTTP/1.1" status: 204 len: 168 time: 0.4282770 May 20 04:22:09.524661 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-f59e3df7-daf1-48c8-8662-386527569c4d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:09.532487 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-a253eddc-9e40-4c7f-b691-30b352c841ac admin admin] 10.222.0.39 "DELETE /v2.0/subnets/7a7294d2-afd0-4235-a9fe-0b3c672f13bc HTTP/1.1" status: 204 len: 168 time: 0.7294006 May 20 04:22:09.535583 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 33446) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:09.542690 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9fb3fca5-141d-4af5-82e9-55f49eeb933c admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=0b1d85f7-0557-4d6c-b22a-e73686fa56e4,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:09.545675 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9fb3fca5-141d-4af5-82e9-55f49eeb933c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.021s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:09.580206 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-86cb0c0f-5abe-4022-99c2-866cc99bf3a7 req-b6832f1b-a4b7-44d6-8265-04ef334b7143 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:09.581684 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-86cb0c0f-5abe-4022-99c2-866cc99bf3a7 req-b6832f1b-a4b7-44d6-8265-04ef334b7143 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 943 time: 0.1184039 May 20 04:22:09.742148 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-86cb0c0f-5abe-4022-99c2-866cc99bf3a7 req-36d8a450-ba14-4f1f-88dc-f0d855f53db9 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1546280 May 20 04:22:09.748608 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-28f5bfec-1fee-4633-ba7c-55adffc0b81c req-07d7ff71-35ee-440a-91b2-f3f4785d6ea4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 210 time: 0.0478823 May 20 04:22:09.791909 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e08c6e49-4b4f-4617-9409-469fe9ed7de1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-593607'] for network, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:09.902236 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cca1d500-40b0-48cb-b1e7-934c0a852290 req-888d1070-fe4d-401d-84eb-7cd2d5352a9d tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 210 time: 0.0426719 May 20 04:22:09.925062 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e08c6e49-4b4f-4617-9409-469fe9ed7de1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-7789108', 'neutron.services.segments.db._delete_segments_for_network-8760688958082'] for network, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:09.933308 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-e08c6e49-4b4f-4617-9409-469fe9ed7de1 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['3455067b-ac51-440f-a307-32c101262582']}} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:22:09.949296 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-e08c6e49-4b4f-4617-9409-469fe9ed7de1 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'd7c9e7f1-8fb6-4935-bdee-61056fc295ff') {'for_net_delete': True} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:22:09.949585 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-e08c6e49-4b4f-4617-9409-469fe9ed7de1 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'd7c9e7f1-8fb6-4935-bdee-61056fc295ff') {} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:22:09.961022 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e08c6e49-4b4f-4617-9409-469fe9ed7de1 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8760688968331', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-853278'] for segment, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:09.977175 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e08c6e49-4b4f-4617-9409-469fe9ed7de1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:09.985578 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-e08c6e49-4b4f-4617-9409-469fe9ed7de1 admin admin] Releasing vlan 823 on physical network physnet1 to pool {{(pid=91644) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 20 04:22:09.986219 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e08c6e49-4b4f-4617-9409-469fe9ed7de1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2001055'] for segment, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: Traceback (most recent call last): May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: timer() May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: cb(*args, **kw) May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: return func(*args, **kwargs) May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: self._notify() May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: self.callback(batched_events) May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: event.method(event) May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: aggregate_id = self._get_aggregate_id(event.segment_id) May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: aggregate_uuid = self.p_client.list_aggregates( May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: return f(self, *a, **k) May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: return self._get(url).json() May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: return self._client.get(url, endpoint_filter=self._ks_filter, May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: return self.request(url, 'GET', **kwargs) May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: base_url = self.get_endpoint(auth, allow=allow, May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: auth = self._auth_required(auth, 'determine endpoint URL') May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 20 04:22:10.000371 dv-cinder-782405-9-iscsi neutron-server[91644]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 20 04:22:10.076341 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e08c6e49-4b4f-4617-9409-469fe9ed7de1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-7789117', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-8053321'] for network, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:10.080599 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-0032ab1e-cb22-44a2-b7d7-fd7de7bae889 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:10.098965 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e08c6e49-4b4f-4617-9409-469fe9ed7de1 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:10.109856 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e08c6e49-4b4f-4617-9409-469fe9ed7de1 admin admin] Pushing event deleted for resources: {'Network': ['ID=3455067b-ac51-440f-a307-32c101262582,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:10.114303 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e08c6e49-4b4f-4617-9409-469fe9ed7de1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.034s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:10.116878 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-e08c6e49-4b4f-4617-9409-469fe9ed7de1 admin admin] 10.222.0.39 "DELETE /v2.0/networks/3455067b-ac51-440f-a307-32c101262582 HTTP/1.1" status: 204 len: 168 time: 0.5802820 May 20 04:22:10.253860 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 33476) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:10.315342 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-685d54ab-b27f-49cb-9127-1a30da6b98ed admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=610f82a3f8c9479b83711aad8d2a1ca5&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0585055 May 20 04:22:10.316258 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 33484) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:10.374565 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-53529cea-79d7-49fd-8012-eb5cd87eeef1 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:10.400203 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-53529cea-79d7-49fd-8012-eb5cd87eeef1 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:10.443134 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0c1c0b08-3351-4847-8905-ed38cd8fe599 req-6b8f4f39-e981-48c6-9573-06a667f2aed3 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:10.444639 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0c1c0b08-3351-4847-8905-ed38cd8fe599 req-6b8f4f39-e981-48c6-9573-06a667f2aed3 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 944 time: 0.0818965 May 20 04:22:10.454476 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-53529cea-79d7-49fd-8012-eb5cd87eeef1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:10.460297 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:10.470833 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-53529cea-79d7-49fd-8012-eb5cd87eeef1 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=14b80be6-20fe-4dd3-860a-d08c679e3978,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:10.473857 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-53529cea-79d7-49fd-8012-eb5cd87eeef1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.013s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:10.479993 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-53529cea-79d7-49fd-8012-eb5cd87eeef1 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:10.483592 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-53529cea-79d7-49fd-8012-eb5cd87eeef1 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/14b80be6-20fe-4dd3-860a-d08c679e3978 HTTP/1.1" status: 204 len: 168 time: 0.1639543 May 20 04:22:10.507497 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0c1c0b08-3351-4847-8905-ed38cd8fe599 req-880c4b3e-89ed-4a7a-93d8-a81df80e2c9f tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0551996 May 20 04:22:10.778881 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 33504) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:10.829610 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-a19a0fc2-1761-4db1-a010-c94aa140c157 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=5d94f8f158c1463ab47e1dfd2ae612a3&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0494983 May 20 04:22:10.832347 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 33512) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:10.963759 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-682c1314-b38d-4a5f-a97e-5d5ebd62b794 req-e4b05efa-aed2-4482-90fd-aa0076c8397f tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 210 time: 0.0858324 May 20 04:22:10.992643 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f110b380-95d7-47b8-912f-f6a48c2d1b64 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:11.027556 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f110b380-95d7-47b8-912f-f6a48c2d1b64 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:11.048717 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-76cbea31-46a2-4096-85a7-2d14e4cf244e req-6920cf0f-2fde-4925-8960-967617b7c5a9 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:11.050186 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-76cbea31-46a2-4096-85a7-2d14e4cf244e req-6920cf0f-2fde-4925-8960-967617b7c5a9 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 943 time: 0.1750753 May 20 04:22:11.137382 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ded1c82c-99a3-4df1-98c9-efe83b33a9a4 req-6ac20294-2544-4aff-bb12-7e73a0e90b90 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 210 time: 0.0530138 May 20 04:22:11.157424 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-76cbea31-46a2-4096-85a7-2d14e4cf244e req-a1660e0f-ec47-4cf0-8f93-e868bc7fe01c tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1004353 May 20 04:22:11.168714 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f110b380-95d7-47b8-912f-f6a48c2d1b64 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:11.174550 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:11.185159 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f110b380-95d7-47b8-912f-f6a48c2d1b64 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=093f0751-ee16-49cc-b807-ef912b257abc,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:11.188048 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-f110b380-95d7-47b8-912f-f6a48c2d1b64 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.013s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:11.194810 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f110b380-95d7-47b8-912f-f6a48c2d1b64 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:11.198436 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-f110b380-95d7-47b8-912f-f6a48c2d1b64 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/093f0751-ee16-49cc-b807-ef912b257abc HTTP/1.1" status: 204 len: 168 time: 0.3625450 May 20 04:22:11.745589 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-034f1eee-144b-49ca-af9a-0682bc622247 req-2a1c7f37-dc5a-43be-a94f-86caf004d4a9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.74 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=9fafdf3a29024ac588e5b24719735677 HTTP/1.1" status: 200 len: 379 time: 0.1786401 May 20 04:22:11.776463 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7e4514b8-15f9-48ca-a84e-d6fc1e459c35 req-635a7328-ddc9-437d-ad58-97fd7a8ef054 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:11.778011 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7e4514b8-15f9-48ca-a84e-d6fc1e459c35 req-635a7328-ddc9-437d-ad58-97fd7a8ef054 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 944 time: 0.1364734 May 20 04:22:11.841499 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7e4514b8-15f9-48ca-a84e-d6fc1e459c35 req-c6636aac-774f-45a0-8633-01f4abf06308 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0564907 May 20 04:22:11.849549 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-51cb7c22-eb82-45b1-9eb0-c244ee77496b req-f1b64b92-2025-4219-9877-7bc81e096012 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:11.851373 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-51cb7c22-eb82-45b1-9eb0-c244ee77496b req-f1b64b92-2025-4219-9877-7bc81e096012 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=0b29d684ade544028fa17828cde46d56&device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 1096 time: 0.2354221 May 20 04:22:11.865380 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-35202614-be1c-4dd4-8e45-1833e64b512f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:11.867307 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-35202614-be1c-4dd4-8e45-1833e64b512f service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=51fcbc78284541db832e4780e76158af&device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 1097 time: 0.1834579 May 20 04:22:11.888356 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [req-034f1eee-144b-49ca-af9a-0682bc622247 req-32c593ea-bfd1-444e-8c09-34ea7e26c59a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Request body: {'port': {'device_id': 'c2ca950f-78ff-4e44-9231-e8e19f4aee5d', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'admin_state_up': True, 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'security_groups': ['202eefda-65f8-4304-8243-021366a25be9']}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:22:12.030047 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-d079804f-1218-4920-a42d-69d7af26f9de service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.14&port_id=849868be-af51-4967-9a6c-93d23cecb450 HTTP/1.1" status: 200 len: 217 time: 0.1094847 May 20 04:22:12.226114 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-392eec25-3357-41bf-9106-5f940536ebd1 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:12.226114 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-392eec25-3357-41bf-9106-5f940536ebd1 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=0f918085-8d33-4e36-a59e-7f270a2b1874 HTTP/1.1" status: 200 len: 855 time: 0.0801134 May 20 04:22:12.245055 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [req-034f1eee-144b-49ca-af9a-0682bc622247 req-32c593ea-bfd1-444e-8c09-34ea7e26c59a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:22:12.256488 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-034f1eee-144b-49ca-af9a-0682bc622247 req-32c593ea-bfd1-444e-8c09-34ea7e26c59a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Usage tracker for resource:port and tenant:9fafdf3a29024ac588e5b24719735677 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:22:12.287197 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-983be3ff-f8cb-420c-8175-4ac6cf3d19aa service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=54fc8d38-193b-49c4-b873-c5353723791f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0534561 May 20 04:22:12.313714 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-034f1eee-144b-49ca-af9a-0682bc622247 req-32c593ea-bfd1-444e-8c09-34ea7e26c59a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Quota usage for port was recalculated. Used quota:3. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:22:12.339629 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-51cb7c22-eb82-45b1-9eb0-c244ee77496b req-2c9bc803-36d2-4251-ada3-5a3525c6aa67 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:12.341081 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-51cb7c22-eb82-45b1-9eb0-c244ee77496b req-2c9bc803-36d2-4251-ada3-5a3525c6aa67 service neutron] 10.222.0.74 "GET /v2.0/networks?tenant_id=0b29d684ade544028fa17828cde46d56&shared=False HTTP/1.1" status: 200 len: 911 time: 0.4520917 May 20 04:22:12.384961 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [req-034f1eee-144b-49ca-af9a-0682bc622247 req-32c593ea-bfd1-444e-8c09-34ea7e26c59a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:22:12.393590 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-93be72ce-b779-41ae-8980-3ffca0b3d76b req-63139318-a2e9-4a0f-8075-a9dbba937645 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 210 time: 0.1320424 May 20 04:22:12.411175 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a6c52d2a-6a9b-4ded-9437-5e4dabf5d9be req-a11fd1ed-737c-4d28-8e7a-1ca507117c87 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 210 time: 0.0734136 May 20 04:22:12.440516 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-034f1eee-144b-49ca-af9a-0682bc622247 req-32c593ea-bfd1-444e-8c09-34ea7e26c59a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Made reservation on behalf of 9fafdf3a29024ac588e5b24719735677 for: {'port': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:22:12.444213 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-034f1eee-144b-49ca-af9a-0682bc622247 req-32c593ea-bfd1-444e-8c09-34ea7e26c59a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:12.583508 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-4fd548ca-18e4-4cee-b462-ecd4a25e4f54 service neutron] 10.222.0.74 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2871199 May 20 04:22:12.619602 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-a9f9c13b-092d-4d2c-b581-10191d2c557d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:12.660727 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-51cb7c22-eb82-45b1-9eb0-c244ee77496b req-29fc3526-61df-4d36-b96c-33825aee6277 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:12.662524 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-51cb7c22-eb82-45b1-9eb0-c244ee77496b req-29fc3526-61df-4d36-b96c-33825aee6277 service neutron] 10.222.0.74 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.3124435 May 20 04:22:12.724489 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d88acfa2-e926-4e5e-9e7b-f0dd9267354a req-8ebf8449-cd88-4918-8b32-e5f5f5afb624 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:12.733197 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d88acfa2-e926-4e5e-9e7b-f0dd9267354a req-8ebf8449-cd88-4918-8b32-e5f5f5afb624 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 943 time: 0.1853766 May 20 04:22:12.738994 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [req-034f1eee-144b-49ca-af9a-0682bc622247 req-32c593ea-bfd1-444e-8c09-34ea7e26c59a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Allocated IP 172.20.0.6 (64bcea8a-6d35-406b-a823-fa1185008174/203a3962-2282-4f51-ac92-c9695b5c0cfe/535d5637-ae66-43cb-bd16-3543bd3b7a8b) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:22:12.816919 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-310138b5-6cfc-42a6-9080-176c026cc0f3 req-b1386c1a-3b9c-480b-b1e3-2c37169a933e service neutron] 10.222.0.74 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2238224 May 20 04:22:12.818346 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d88acfa2-e926-4e5e-9e7b-f0dd9267354a req-5a279f4b-718b-46a3-880a-4cd1d0a9e31f tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0842266 May 20 04:22:13.126959 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-034f1eee-144b-49ca-af9a-0682bc622247 req-32c593ea-bfd1-444e-8c09-34ea7e26c59a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:13.145439 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-66adb1fd-065b-4b9b-9115-93a704afe260 req-8458d6b5-6b45-45af-88fe-603c3b0b512e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:13.152949 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-66adb1fd-065b-4b9b-9115-93a704afe260 req-8458d6b5-6b45-45af-88fe-603c3b0b512e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 944 time: 0.1657302 May 20 04:22:13.180356 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.provisioning_blocks [req-034f1eee-144b-49ca-af9a-0682bc622247 req-32c593ea-bfd1-444e-8c09-34ea7e26c59a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Transition to ACTIVE for port object 535d5637-ae66-43cb-bd16-3543bd3b7a8b will not be triggered until provisioned by entity DHCP. {{(pid=91643) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:22:13.265042 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-66adb1fd-065b-4b9b-9115-93a704afe260 req-36d109e4-f9d8-4e2f-b68c-5d5267e4111a tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1112721 May 20 04:22:13.299680 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-034f1eee-144b-49ca-af9a-0682bc622247 req-32c593ea-bfd1-444e-8c09-34ea7e26c59a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:13.312255 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:13.374502 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-7d9e1d1f-fe65-419a-9ffb-eeda9a8b4f64 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:13.375333 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-034f1eee-144b-49ca-af9a-0682bc622247 req-32c593ea-bfd1-444e-8c09-34ea7e26c59a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Network 64bcea8a-6d35-406b-a823-fa1185008174 is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:22:13.375798 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-7d9e1d1f-fe65-419a-9ffb-eeda9a8b4f64 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.74 "GET /v2.0/networks?id=36b37213-8439-447d-9eba-a78baf7c698e HTTP/1.1" status: 200 len: 819 time: 0.5376916 May 20 04:22:13.383293 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [req-034f1eee-144b-49ca-af9a-0682bc622247 req-32c593ea-bfd1-444e-8c09-34ea7e26c59a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:22:13.390538 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [req-034f1eee-144b-49ca-af9a-0682bc622247 req-32c593ea-bfd1-444e-8c09-34ea7e26c59a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:22:13.395152 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-1d1804cc-6732-4c5f-8995-aaa42d5e5265 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Request body: {'port': {'device_id': 'bbf6ae7d-8ff5-4b69-adef-47220d6c5c93', 'network_id': '36b37213-8439-447d-9eba-a78baf7c698e', 'admin_state_up': True, 'tenant_id': 'bec56d8703cf47f3b7193b7d1f55396d'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:22:13.398612 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-034f1eee-144b-49ca-af9a-0682bc622247 req-32c593ea-bfd1-444e-8c09-34ea7e26c59a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:13.399190 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-034f1eee-144b-49ca-af9a-0682bc622247 req-32c593ea-bfd1-444e-8c09-34ea7e26c59a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:13.428920 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-32c593ea-bfd1-444e-8c09-34ea7e26c59a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Pushing event updated for resources: {'Port': ['ID=535d5637-ae66-43cb-bd16-3543bd3b7a8b,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:13.432909 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-32c593ea-bfd1-444e-8c09-34ea7e26c59a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.121s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:13.447076 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-034f1eee-144b-49ca-af9a-0682bc622247 req-32c593ea-bfd1-444e-8c09-34ea7e26c59a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.5649993 May 20 04:22:13.458795 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Request body: {'port': {'device_id': 'c2ca950f-78ff-4e44-9231-e8e19f4aee5d', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:22:13.531281 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-02066f98-7f68-4d15-9096-a80e0b6a8a26 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:13.533710 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-02066f98-7f68-4d15-9096-a80e0b6a8a26 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=0b29d684ade544028fa17828cde46d56&device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 1096 time: 0.5794418 May 20 04:22:13.674602 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-f3c7d8c2-87df-4046-bf55-0993c2b9a20c service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.2&port_id=efb81444-f373-4e9c-a0aa-e3270bd4b980 HTTP/1.1" status: 200 len: 217 time: 0.0975604 May 20 04:22:13.766872 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-73ceee33-e561-407c-b433-ca0d2875e272 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:13.768468 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-73ceee33-e561-407c-b433-ca0d2875e272 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=b9f2dbbb-b724-44b9-878b-2267c4d95aee HTTP/1.1" status: 200 len: 850 time: 0.0862219 May 20 04:22:13.770918 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-1d1804cc-6732-4c5f-8995-aaa42d5e5265 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:22:13.815834 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.74', 56967) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:13.816770 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-1d1804cc-6732-4c5f-8995-aaa42d5e5265 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attempting to reserve 1 items for resource port. Total usage: 1; quota limit: 500; headroom:499 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:22:13.849401 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-52b5bf35-eab1-4bf6-a01e-842e537dda25 None None] Provisioning for port 535d5637-ae66-43cb-bd16-3543bd3b7a8b completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:22:13.855095 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-52b5bf35-eab1-4bf6-a01e-842e537dda25 None None] Provisioning complete for port 535d5637-ae66-43cb-bd16-3543bd3b7a8b triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:22:13.858843 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-52b5bf35-eab1-4bf6-a01e-842e537dda25 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:13.941058 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e942d6bf-bbc1-4e29-8619-0e858bdba964 req-efef0ce8-fce1-4f86-b0c1-e1092a5288d8 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 210 time: 0.1740842 May 20 04:22:13.982632 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-52b5bf35-eab1-4bf6-a01e-842e537dda25 None None] Port 535d5637-ae66-43cb-bd16-3543bd3b7a8b cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:22:14.040818 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-3987311a-eb3f-4ede-a3fa-f3dc03833c40 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=a009b619-8ab3-40e3-b5f3-9cc97ccd3415&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.2619121 May 20 04:22:14.103154 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:14.109762 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-1d1804cc-6732-4c5f-8995-aaa42d5e5265 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Made reservation on behalf of bec56d8703cf47f3b7193b7d1f55396d for: {'port': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:22:14.113615 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-1d1804cc-6732-4c5f-8995-aaa42d5e5265 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:14.357339 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0c119fd2-e50a-4cbe-a3b7-e5d6b95ef2b0 req-4aed61a0-03df-417a-b4d0-c13fbac9e886 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:14.358300 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-0c119fd2-e50a-4cbe-a3b7-e5d6b95ef2b0 req-4aed61a0-03df-417a-b4d0-c13fbac9e886 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Synchronizing usage tracker for tenant:9fafdf3a29024ac588e5b24719735677 on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:22:14.403320 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-90f9dea1-4d4f-49b7-b3d0-ec345cc1d0ee tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:14.405086 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-90f9dea1-4d4f-49b7-b3d0-ec345cc1d0ee tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.74 "GET /v2.0/networks?id=36b37213-8439-447d-9eba-a78baf7c698e HTTP/1.1" status: 200 len: 819 time: 0.5811627 May 20 04:22:14.408066 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-1d1804cc-6732-4c5f-8995-aaa42d5e5265 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Allocated IP 172.20.0.8 (36b37213-8439-447d-9eba-a78baf7c698e/b25ae9c8-e11d-45ee-bde2-332cc1538602/706277c6-cd8c-4146-ab91-ed9504446690) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:22:14.421337 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-a877c026-2849-4546-84c2-b96c8bbad20f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Request body: {'port': {'device_id': '0d234180-40a1-4229-990e-f796674056ce', 'network_id': '36b37213-8439-447d-9eba-a78baf7c698e', 'admin_state_up': True, 'tenant_id': 'bec56d8703cf47f3b7193b7d1f55396d'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:22:14.462257 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-0c119fd2-e50a-4cbe-a3b7-e5d6b95ef2b0 req-4aed61a0-03df-417a-b4d0-c13fbac9e886 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Unset dirty status for tenant:9fafdf3a29024ac588e5b24719735677 on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:22:14.466528 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0c119fd2-e50a-4cbe-a3b7-e5d6b95ef2b0 req-4aed61a0-03df-417a-b4d0-c13fbac9e886 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 931 time: 0.6921501 May 20 04:22:14.479961 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3409bf40-c31f-4b21-8754-a0f87946a954 req-c7371f04-a46b-40aa-b427-e8cbe4066551 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:14.481621 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3409bf40-c31f-4b21-8754-a0f87946a954 req-c7371f04-a46b-40aa-b427-e8cbe4066551 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 943 time: 0.3796046 May 20 04:22:14.493313 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 33682) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:14.506476 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-3f5a2b06-2b64-48cf-b884-13de97def8cd service neutron] 10.222.0.74 "GET /v2.0/networks/a009b619-8ab3-40e3-b5f3-9cc97ccd3415?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.4481044 May 20 04:22:14.643794 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0c119fd2-e50a-4cbe-a3b7-e5d6b95ef2b0 req-73d95778-1d2d-4c20-a54b-910644699c45 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1608589 May 20 04:22:14.712818 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3409bf40-c31f-4b21-8754-a0f87946a954 req-4940dcc3-9f18-430d-b072-1849df021d2e tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2166290 May 20 04:22:14.715905 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3835b7df-d700-4e40-817f-5e345634cc24 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:14.762933 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-3835b7df-d700-4e40-817f-5e345634cc24 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:22:14.773473 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-3835b7df-d700-4e40-817f-5e345634cc24 admin admin] Usage tracker for resource:security_group_rule and tenant:6aa50f056bba4be89aafcbde64ae4190 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:22:14.806284 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-3835b7df-d700-4e40-817f-5e345634cc24 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:22:14.825523 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-3835b7df-d700-4e40-817f-5e345634cc24 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:22:14.865703 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 33700) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:14.889730 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-72fb1445-b558-4a42-8fb8-0c97e550b2aa req-48d64a95-1cfb-46e7-8261-28f755d048a9 service neutron] 10.222.0.74 "GET /v2.0/networks/a009b619-8ab3-40e3-b5f3-9cc97ccd3415?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3738344 May 20 04:22:14.908546 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-a877c026-2849-4546-84c2-b96c8bbad20f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:22:14.919093 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-a877c026-2849-4546-84c2-b96c8bbad20f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Usage tracker for resource:port and tenant:bec56d8703cf47f3b7193b7d1f55396d is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:22:14.924121 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-a877c026-2849-4546-84c2-b96c8bbad20f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Quota usage for port was recalculated. Used quota:1. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:22:14.933785 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-a877c026-2849-4546-84c2-b96c8bbad20f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:22:14.950053 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 535d5637-ae66-43cb-bd16-3543bd3b7a8b {{(pid=91643) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:22:14.959483 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:14.969750 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-a877c026-2849-4546-84c2-b96c8bbad20f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Made reservation on behalf of bec56d8703cf47f3b7193b7d1f55396d for: {'port': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:22:14.973631 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-a877c026-2849-4546-84c2-b96c8bbad20f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:14.975896 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:14.992309 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c20e0f2c-58e1-4adb-a699-94299e73e15f req-2265b869-0f28-4c0a-bf8d-65a5491a30ca tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:14.993805 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c20e0f2c-58e1-4adb-a699-94299e73e15f req-2265b869-0f28-4c0a-bf8d-65a5491a30ca tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 944 time: 0.2268472 May 20 04:22:14.997185 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.provisioning_blocks [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Transition to ACTIVE for port object 535d5637-ae66-43cb-bd16-3543bd3b7a8b will not be triggered until provisioned by entity L2. {{(pid=91643) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:22:15.026446 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:15.042763 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-4352e816-0ed1-4bc4-a6ce-54c93f12f036 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:15.104582 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-1fad8f72-72a4-4531-8074-408c11473788 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:15.117540 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-1d1804cc-6732-4c5f-8995-aaa42d5e5265 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:15.223931 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-1d1804cc-6732-4c5f-8995-aaa42d5e5265 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:15.246166 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:15.286096 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7711194d-933c-413d-b7b2-59b89552c2c9 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:15.315595 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d1a18296-a381-4793-974c-b489ea3e8eca req-4c1555a3-7929-4773-9c6e-7adff34c0c21 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 210 time: 0.2074158 May 20 04:22:15.337637 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-1d1804cc-6732-4c5f-8995-aaa42d5e5265 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Network 36b37213-8439-447d-9eba-a78baf7c698e is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:22:15.343018 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-1d1804cc-6732-4c5f-8995-aaa42d5e5265 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:22:15.350388 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-1d1804cc-6732-4c5f-8995-aaa42d5e5265 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:22:15.377832 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c20e0f2c-58e1-4adb-a699-94299e73e15f req-d2899626-bb83-4da2-a9a9-668f5816be75 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.3748937 May 20 04:22:15.382403 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-7711194d-933c-413d-b7b2-59b89552c2c9 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:22:15.387927 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-7711194d-933c-413d-b7b2-59b89552c2c9 admin admin] Usage tracker for resource:security_group_rule and tenant:2a1490b39ada451abb474b9314585f8d is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:22:15.395293 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-7711194d-933c-413d-b7b2-59b89552c2c9 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:22:15.402507 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-7711194d-933c-413d-b7b2-59b89552c2c9 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:22:15.405889 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3835b7df-d700-4e40-817f-5e345634cc24 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:15.412406 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 33718) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:15.413374 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Pushing event updated for resources: {'Port': ['ID=535d5637-ae66-43cb-bd16-3543bd3b7a8b,revision_number=2']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:15.423667 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.381s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:15.436928 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3835b7df-d700-4e40-817f-5e345634cc24 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:15.455117 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-a877c026-2849-4546-84c2-b96c8bbad20f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Allocated IP 172.20.0.13 (36b37213-8439-447d-9eba-a78baf7c698e/b25ae9c8-e11d-45ee-bde2-332cc1538602/a8b0c56a-171d-40e9-beb7-c7b63fa031b9) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:22:15.456973 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-db98a709-3ab7-4fab-a7ba-f968caf630b9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:15.486238 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1d1804cc-6732-4c5f-8995-aaa42d5e5265 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Pushing event updated for resources: {'Port': ['ID=706277c6-cd8c-4146-ab91-ed9504446690,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:15.491294 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Attempting to bind port 535d5637-ae66-43cb-bd16-3543bd3b7a8b on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:22:15.491648 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Attempting to bind port 535d5637-ae66-43cb-bd16-3543bd3b7a8b by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': '9d1b8515-c35d-48e6-86ad-4eff412e924a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 847, 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174'}] {{(pid=91643) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:22:15.492026 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Attempting to bind port 535d5637-ae66-43cb-bd16-3543bd3b7a8b on network 64bcea8a-6d35-406b-a823-fa1185008174 {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:22:15.492302 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-1d1804cc-6732-4c5f-8995-aaa42d5e5265 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.246s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:15.511482 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Port 535d5637-ae66-43cb-bd16-3543bd3b7a8b on network 64bcea8a-6d35-406b-a823-fa1185008174 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:22:15.511814 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Attempting to bind port 535d5637-ae66-43cb-bd16-3543bd3b7a8b on network 64bcea8a-6d35-406b-a823-fa1185008174 {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:22:15.513812 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3835b7df-d700-4e40-817f-5e345634cc24 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=0a93e254-06ba-4a4d-9aa8-fde8ef2a7c1f,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:15.524042 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-3835b7df-d700-4e40-817f-5e345634cc24 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.067s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:15.528261 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 22, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:22:15.528662 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Checking segment: {'id': '9d1b8515-c35d-48e6-86ad-4eff412e924a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 847, 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91643) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:22:15.529288 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Bound using segment: {'id': '9d1b8515-c35d-48e6-86ad-4eff412e924a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 847, 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174'} {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:22:15.529644 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Bound port: 535d5637-ae66-43cb-bd16-3543bd3b7a8b, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '9d1b8515-c35d-48e6-86ad-4eff412e924a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 847, 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174'}}] {{(pid=91643) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:22:15.529982 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:15.536391 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-1d1804cc-6732-4c5f-8995-aaa42d5e5265 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:15.536999 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-1d1804cc-6732-4c5f-8995-aaa42d5e5265 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:15.545458 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-3835b7df-d700-4e40-817f-5e345634cc24 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=6aa50f056bba4be89aafcbde64ae4190&name=default HTTP/1.1" status: 200 len: 2979 time: 1.0495770 May 20 04:22:15.548484 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 33724) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:15.601739 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-1d1804cc-6732-4c5f-8995-aaa42d5e5265 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 2.2116680 May 20 04:22:15.656094 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-70c1bf19-d53f-428e-996f-0e8fa9f8db06 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:15.707893 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-70c1bf19-d53f-428e-996f-0e8fa9f8db06 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:22:15.716393 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-70c1bf19-d53f-428e-996f-0e8fa9f8db06 admin admin] Usage tracker for resource:security_group_rule and tenant:54e4842d507d4d88b60082dfaf35db3b is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:22:15.720583 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-70c1bf19-d53f-428e-996f-0e8fa9f8db06 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:22:15.726703 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-a877c026-2849-4546-84c2-b96c8bbad20f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:15.751876 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-70c1bf19-d53f-428e-996f-0e8fa9f8db06 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:22:15.766421 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] For port 535d5637-ae66-43cb-bd16-3543bd3b7a8b, host hv-cinder-78240, cleared binding levels {{(pid=91643) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:22:15.769183 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-68ac8851-9d6c-4364-875d-1b8c3e6cbc0e admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:15.771959 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-a877c026-2849-4546-84c2-b96c8bbad20f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:15.779574 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:15.792077 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7711194d-933c-413d-b7b2-59b89552c2c9 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:15.843872 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] For port 535d5637-ae66-43cb-bd16-3543bd3b7a8b, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=535d5637-ae66-43cb-bd16-3543bd3b7a8b,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91643) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:22:15.851628 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-a877c026-2849-4546-84c2-b96c8bbad20f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Network 36b37213-8439-447d-9eba-a78baf7c698e is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:22:15.854674 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-a877c026-2849-4546-84c2-b96c8bbad20f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:22:15.858560 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-a877c026-2849-4546-84c2-b96c8bbad20f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:22:15.887809 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a877c026-2849-4546-84c2-b96c8bbad20f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Pushing event updated for resources: {'Port': ['ID=a8b0c56a-171d-40e9-beb7-c7b63fa031b9,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:15.889559 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7711194d-933c-413d-b7b2-59b89552c2c9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:15.894029 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-a877c026-2849-4546-84c2-b96c8bbad20f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:15.910949 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:15.921165 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-68ac8851-9d6c-4364-875d-1b8c3e6cbc0e admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:15.923127 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-a877c026-2849-4546-84c2-b96c8bbad20f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:15.923755 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-a877c026-2849-4546-84c2-b96c8bbad20f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:15.927700 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-6553cfb0-7b48-43db-8efe-6b5b9c3132b3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:15.941592 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-a877c026-2849-4546-84c2-b96c8bbad20f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 1.5269573 May 20 04:22:16.109207 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7711194d-933c-413d-b7b2-59b89552c2c9 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=16faf1c7-760d-4867-aad6-17719c3e96b1,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:16.116163 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-7711194d-933c-413d-b7b2-59b89552c2c9 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=2a1490b39ada451abb474b9314585f8d&name=default HTTP/1.1" status: 200 len: 2979 time: 1.2448053 May 20 04:22:16.116163 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:16.116163 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:16.117325 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 33732) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:16.117475 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-034f1eee-144b-49ca-af9a-0682bc622247 req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] 10.222.0.74 "PUT /v2.0/ports/535d5637-ae66-43cb-bd16-3543bd3b7a8b HTTP/1.1" status: 200 len: 1093 time: 2.6643217 May 20 04:22:16.120381 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-7711194d-933c-413d-b7b2-59b89552c2c9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.193s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:16.122269 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'c2ca950f-78ff-4e44-9231-e8e19f4aee5d', 'tag': '535d5637-ae66-43cb-bd16-3543bd3b7a8b'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:22:16.160032 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:16.176471 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.150s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:16.207518 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-2643d934-63c1-44c7-84ec-c9d96327bdd3 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:16.239527 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-2643d934-63c1-44c7-84ec-c9d96327bdd3 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:16.301440 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-68ac8851-9d6c-4364-875d-1b8c3e6cbc0e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:16.315009 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-2643d934-63c1-44c7-84ec-c9d96327bdd3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:16.322238 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-b353e2ee-0460-4b59-b1e3-4f699d365142 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:16.322698 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Pushing event updated for resources: {'Port': ['ID=535d5637-ae66-43cb-bd16-3543bd3b7a8b,revision_number=3']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:16.332241 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a4566591-0318-4571-b30e-7bb74233ab76 req-fa9508f4-11cc-4266-a687-34fe2770ab72 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:16.333727 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a4566591-0318-4571-b30e-7bb74233ab76 req-fa9508f4-11cc-4266-a687-34fe2770ab72 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.5586996 May 20 04:22:16.334566 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.158s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:16.339036 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.017s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:16.340938 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2643d934-63c1-44c7-84ec-c9d96327bdd3 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=16faf1c7-760d-4867-aad6-17719c3e96b1,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:16.344611 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-2643d934-63c1-44c7-84ec-c9d96327bdd3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:16.347046 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-2643d934-63c1-44c7-84ec-c9d96327bdd3 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:16.348490 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-2643d934-63c1-44c7-84ec-c9d96327bdd3 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/16faf1c7-760d-4867-aad6-17719c3e96b1 HTTP/1.1" status: 204 len: 168 time: 0.2296205 May 20 04:22:16.353645 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a00602d7-882e-421a-9d95-b138e5d1c4be req-a5f8a6ff-0ba6-4d74-a645-183ed5ed4d59 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:16.355300 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a00602d7-882e-421a-9d95-b138e5d1c4be req-a5f8a6ff-0ba6-4d74-a645-183ed5ed4d59 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 943 time: 0.5226419 May 20 04:22:16.365813 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-70c1bf19-d53f-428e-996f-0e8fa9f8db06 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:16.383993 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-68ac8851-9d6c-4364-875d-1b8c3e6cbc0e admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=0a93e254-06ba-4a4d-9aa8-fde8ef2a7c1f,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:16.408175 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-68ac8851-9d6c-4364-875d-1b8c3e6cbc0e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.068s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:16.408871 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-70c1bf19-d53f-428e-996f-0e8fa9f8db06 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:16.434111 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-c16dac2a-83e9-447f-8b47-8c285f12960c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:16.483179 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-68ac8851-9d6c-4364-875d-1b8c3e6cbc0e admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:16.484299 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-68ac8851-9d6c-4364-875d-1b8c3e6cbc0e admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/0a93e254-06ba-4a4d-9aa8-fde8ef2a7c1f HTTP/1.1" status: 204 len: 168 time: 0.9332757 May 20 04:22:16.503541 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a00602d7-882e-421a-9d95-b138e5d1c4be req-3266634a-9114-44d1-9b6b-997ea50b9e59 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1402137 May 20 04:22:16.510638 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a4566591-0318-4571-b30e-7bb74233ab76 req-73de93bf-9fe2-4e3e-acaf-6d0716a76aa7 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1484590 May 20 04:22:16.514191 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-70c1bf19-d53f-428e-996f-0e8fa9f8db06 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=54e4842d507d4d88b60082dfaf35db3b&name=default HTTP/1.1" status: 200 len: 2979 time: 1.0939300 May 20 04:22:16.514947 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-70c1bf19-d53f-428e-996f-0e8fa9f8db06 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=35b4bdfc-85cf-4a73-a098-fc007804d5b4,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:16.517482 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 33752) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:16.518026 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-70c1bf19-d53f-428e-996f-0e8fa9f8db06 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:16.647045 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8b2b1b6d-5a79-44b8-b579-46d4407622d1 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:16.694383 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8b2b1b6d-5a79-44b8-b579-46d4407622d1 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:16.766570 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8b2b1b6d-5a79-44b8-b579-46d4407622d1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:16.773225 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:16.785283 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8b2b1b6d-5a79-44b8-b579-46d4407622d1 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=35b4bdfc-85cf-4a73-a098-fc007804d5b4,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:16.790223 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 33760) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:16.791686 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-8b2b1b6d-5a79-44b8-b579-46d4407622d1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:16.792653 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8b2b1b6d-5a79-44b8-b579-46d4407622d1 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:16.796603 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-8b2b1b6d-5a79-44b8-b579-46d4407622d1 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/35b4bdfc-85cf-4a73-a098-fc007804d5b4 HTTP/1.1" status: 204 len: 168 time: 0.2785604 May 20 04:22:16.820083 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d8952f97-8d25-42e6-9954-747d1e0922a9 req-0cc3948a-e0eb-4617-a971-c81b25a7b1cd tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:16.823727 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d8952f97-8d25-42e6-9954-747d1e0922a9 req-0cc3948a-e0eb-4617-a971-c81b25a7b1cd tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 932 time: 0.1899490 May 20 04:22:16.828885 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-798267d7-8751-47c3-bd5e-7a231208c725 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:16.872686 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-798267d7-8751-47c3-bd5e-7a231208c725 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:22:16.889009 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-798267d7-8751-47c3-bd5e-7a231208c725 admin admin] Usage tracker for resource:security_group_rule and tenant:23cd2df55eb34d6cb3a73cd72718c08d is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:22:16.899345 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c9539ce0-deeb-4658-93d0-f8e1a5e8a424 req-8ee2adf7-34ee-4772-a770-1015edd68868 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:16.901149 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c9539ce0-deeb-4658-93d0-f8e1a5e8a424 req-8ee2adf7-34ee-4772-a770-1015edd68868 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 944 time: 0.1771088 May 20 04:22:16.902114 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-798267d7-8751-47c3-bd5e-7a231208c725 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:22:16.912510 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-798267d7-8751-47c3-bd5e-7a231208c725 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:22:16.916793 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-d8952f97-8d25-42e6-9954-747d1e0922a9 req-a6b03c9d-f69f-46a0-b9b8-fd64f92b0c20 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Synchronizing usage tracker for tenant:bec56d8703cf47f3b7193b7d1f55396d on resource:security_group {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:22:16.984939 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-d8952f97-8d25-42e6-9954-747d1e0922a9 req-a6b03c9d-f69f-46a0-b9b8-fd64f92b0c20 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Unset dirty status for tenant:bec56d8703cf47f3b7193b7d1f55396d on resource:security_group {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:22:16.986445 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d8952f97-8d25-42e6-9954-747d1e0922a9 req-a6b03c9d-f69f-46a0-b9b8-fd64f92b0c20 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1586359 May 20 04:22:16.998781 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c9539ce0-deeb-4658-93d0-f8e1a5e8a424 req-abfcd6c3-2343-4f21-bd32-41bb2545008d tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0768018 May 20 04:22:17.043865 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 33770) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:17.061282 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-552e0c96-f2af-4537-9895-416f719f34d5 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.081095 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-798267d7-8751-47c3-bd5e-7a231208c725 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.084944 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-552e0c96-f2af-4537-9895-416f719f34d5 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:22:17.085102 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-798267d7-8751-47c3-bd5e-7a231208c725 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.089491 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-552e0c96-f2af-4537-9895-416f719f34d5 admin admin] Usage tracker for resource:security_group_rule and tenant:209e78460aa24d2e8c5fb193e865bae6 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:22:17.092749 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-552e0c96-f2af-4537-9895-416f719f34d5 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:22:17.095832 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-f110b380-95d7-47b8-912f-f6a48c2d1b64 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:17.099097 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-552e0c96-f2af-4537-9895-416f719f34d5 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:22:17.151130 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-798267d7-8751-47c3-bd5e-7a231208c725 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=23cd2df55eb34d6cb3a73cd72718c08d&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3568566 May 20 04:22:17.151130 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-798267d7-8751-47c3-bd5e-7a231208c725 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=8f0578b1-32b8-4bc5-906d-4ca4987f1245,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:17.152841 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-798267d7-8751-47c3-bd5e-7a231208c725 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.057s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:17.153564 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 33774) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:17.225870 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-75f8ac58-3baf-40b4-bbd5-d0a729a9374c admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.231786 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-552e0c96-f2af-4537-9895-416f719f34d5 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.236836 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-552e0c96-f2af-4537-9895-416f719f34d5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.246835 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-12233e74-cb7e-48a5-881a-49d4928129bd admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:17.254313 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-75f8ac58-3baf-40b4-bbd5-d0a729a9374c admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.291135 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-552e0c96-f2af-4537-9895-416f719f34d5 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=209e78460aa24d2e8c5fb193e865bae6&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2461975 May 20 04:22:17.291633 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 33778) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:17.293661 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 33782) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:17.304486 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-552e0c96-f2af-4537-9895-416f719f34d5 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=65ef3798-42fc-4d64-b7b7-3ce1f99d3e4c,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:17.309427 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-552e0c96-f2af-4537-9895-416f719f34d5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.063s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:17.315975 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7c6577f3-c9b1-4da8-8aa9-a53681d9cad4 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.329868 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'c2ca950f-78ff-4e44-9231-e8e19f4aee5d', 'tag': '535d5637-ae66-43cb-bd16-3543bd3b7a8b', 'status': 'completed', 'code': 200} May 20 04:22:17.335073 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-7c6577f3-c9b1-4da8-8aa9-a53681d9cad4 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:22:17.339835 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-7c6577f3-c9b1-4da8-8aa9-a53681d9cad4 admin admin] Usage tracker for resource:security_group_rule and tenant:b098221dfa064a448d815989ba40aa81 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:22:17.343292 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-7c6577f3-c9b1-4da8-8aa9-a53681d9cad4 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:22:17.349535 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-7c6577f3-c9b1-4da8-8aa9-a53681d9cad4 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:22:17.357645 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d5777953-7f9d-41d0-b8ec-d6c7583cde32 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.361609 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-75f8ac58-3baf-40b4-bbd5-d0a729a9374c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.367174 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-70c1bf19-d53f-428e-996f-0e8fa9f8db06 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:17.383306 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d5777953-7f9d-41d0-b8ec-d6c7583cde32 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.387959 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-75f8ac58-3baf-40b4-bbd5-d0a729a9374c admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=8f0578b1-32b8-4bc5-906d-4ca4987f1245,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:17.390166 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-75f8ac58-3baf-40b4-bbd5-d0a729a9374c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:17.399046 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-75f8ac58-3baf-40b4-bbd5-d0a729a9374c admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.400328 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-75f8ac58-3baf-40b4-bbd5-d0a729a9374c admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/8f0578b1-32b8-4bc5-906d-4ca4987f1245 HTTP/1.1" status: 204 len: 168 time: 0.2458785 May 20 04:22:17.441434 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d5777953-7f9d-41d0-b8ec-d6c7583cde32 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.447681 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:17.458629 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d5777953-7f9d-41d0-b8ec-d6c7583cde32 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=65ef3798-42fc-4d64-b7b7-3ce1f99d3e4c,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:17.470849 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d5777953-7f9d-41d0-b8ec-d6c7583cde32 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.024s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:17.472719 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7c6577f3-c9b1-4da8-8aa9-a53681d9cad4 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.473187 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d5777953-7f9d-41d0-b8ec-d6c7583cde32 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.474292 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-d5777953-7f9d-41d0-b8ec-d6c7583cde32 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/65ef3798-42fc-4d64-b7b7-3ce1f99d3e4c HTTP/1.1" status: 204 len: 168 time: 0.1797650 May 20 04:22:17.478609 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7c6577f3-c9b1-4da8-8aa9-a53681d9cad4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.490666 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:17.534260 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-7c6577f3-c9b1-4da8-8aa9-a53681d9cad4 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=b098221dfa064a448d815989ba40aa81&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2414963 May 20 04:22:17.535152 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7c6577f3-c9b1-4da8-8aa9-a53681d9cad4 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d1f20654-55fb-4c99-84ca-40fd3594198a,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:17.537088 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 33792) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:17.538151 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-7c6577f3-c9b1-4da8-8aa9-a53681d9cad4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:17.621033 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0c7c941d-f2cc-4283-8527-820027d5a52c admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.650488 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0c7c941d-f2cc-4283-8527-820027d5a52c admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.725983 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0c7c941d-f2cc-4283-8527-820027d5a52c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.738343 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:17.760304 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0c7c941d-f2cc-4283-8527-820027d5a52c admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d1f20654-55fb-4c99-84ca-40fd3594198a,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:17.762359 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0c7c941d-f2cc-4283-8527-820027d5a52c admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.766355 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-0c7c941d-f2cc-4283-8527-820027d5a52c admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/d1f20654-55fb-4c99-84ca-40fd3594198a HTTP/1.1" status: 204 len: 168 time: 0.2269027 May 20 04:22:17.767556 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-0c7c941d-f2cc-4283-8527-820027d5a52c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.029s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:17.798516 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fe39e364-ed2f-4524-9a26-81adca6daab8 req-49e30322-4390-4ca0-927b-82a3750ccdc0 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:17.799957 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fe39e364-ed2f-4524-9a26-81adca6daab8 req-49e30322-4390-4ca0-927b-82a3750ccdc0 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.1436691 May 20 04:22:17.819646 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:17.821603 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b34888a7-148a-49c2-bbe2-9fc4e69c4729 req-aba8e014-b2eb-4b8f-8ba8-fbeb492c727c tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:17.823087 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b34888a7-148a-49c2-bbe2-9fc4e69c4729 req-aba8e014-b2eb-4b8f-8ba8-fbeb492c727c tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 943 time: 0.1281641 May 20 04:22:17.896437 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fe39e364-ed2f-4524-9a26-81adca6daab8 req-dcf5d703-7113-4723-a67d-cbc1c6201da8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0829215 May 20 04:22:17.899117 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b34888a7-148a-49c2-bbe2-9fc4e69c4729 req-588d7a6c-b1a8-4d58-a965-b4a08e788cd1 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0616808 May 20 04:22:18.245580 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1b77af8e-5bcb-4b9b-b43a-8380825396c4 req-34b1fd77-8977-4c3a-8464-e7eb9ebeb30d tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:18.247059 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1b77af8e-5bcb-4b9b-b43a-8380825396c4 req-34b1fd77-8977-4c3a-8464-e7eb9ebeb30d tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 932 time: 0.1418808 May 20 04:22:18.259781 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-329c933e-da4c-4316-aac1-984a8a080b01 req-e331040c-5ac1-4174-b0bf-8ed24c8e9627 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:18.261224 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-329c933e-da4c-4316-aac1-984a8a080b01 req-e331040c-5ac1-4174-b0bf-8ed24c8e9627 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 944 time: 0.1194270 May 20 04:22:18.336428 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1b77af8e-5bcb-4b9b-b43a-8380825396c4 req-97dcede6-d787-4fe4-a3d2-da5280d11a99 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0727658 May 20 04:22:18.350879 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-329c933e-da4c-4316-aac1-984a8a080b01 req-3c8e9d4d-8ebe-4f87-af96-a13482aef007 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0759065 May 20 04:22:19.184373 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b27f4c0e-c427-4165-8b20-a1a6d8c39358 req-d6f18b77-66c3-4d47-bde7-d6b11eff9d8d tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:19.185853 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b27f4c0e-c427-4165-8b20-a1a6d8c39358 req-d6f18b77-66c3-4d47-bde7-d6b11eff9d8d tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 943 time: 0.1579063 May 20 04:22:19.199650 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3ece23c8-e2bd-46b6-8b19-adfd203bfb1d req-994a282c-dc3f-40ea-9be4-90f46d1ae589 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:19.206065 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3ece23c8-e2bd-46b6-8b19-adfd203bfb1d req-994a282c-dc3f-40ea-9be4-90f46d1ae589 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.1406975 May 20 04:22:19.294036 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3ece23c8-e2bd-46b6-8b19-adfd203bfb1d req-f60613a7-0081-4544-972b-023264c718d7 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0870092 May 20 04:22:19.304565 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b27f4c0e-c427-4165-8b20-a1a6d8c39358 req-0f03bf43-9612-48e5-bc43-fc65698330cd tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0958819 May 20 04:22:19.325667 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Request body: {'port': {'device_id': 'bbf6ae7d-8ff5-4b69-adef-47220d6c5c93', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:22:19.398980 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Request body: {'port': {'device_id': '0d234180-40a1-4229-990e-f796674056ce', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:22:19.457865 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-034f1eee-144b-49ca-af9a-0682bc622247 req-423addc9-0a50-43a1-b1d8-ded8effc19ec service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:19.461520 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-034f1eee-144b-49ca-af9a-0682bc622247 req-423addc9-0a50-43a1-b1d8-ded8effc19ec service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=9fafdf3a29024ac588e5b24719735677&device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 1096 time: 0.1254058 May 20 04:22:19.567575 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-034f1eee-144b-49ca-af9a-0682bc622247 req-e6fbb7d4-d809-456d-8ea6-1dafe8fde5a5 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=535d5637-ae66-43cb-bd16-3543bd3b7a8b HTTP/1.1" status: 200 len: 217 time: 0.0993345 May 20 04:22:19.785232 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:19.816868 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:19.937520 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-034f1eee-144b-49ca-af9a-0682bc622247 req-51ae5420-a09f-4986-b94c-d452e3f02198 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:19.939182 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-034f1eee-144b-49ca-af9a-0682bc622247 req-51ae5420-a09f-4986-b94c-d452e3f02198 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=203a3962-2282-4f51-ac92-c9695b5c0cfe HTTP/1.1" status: 200 len: 856 time: 0.3199952 May 20 04:22:19.959165 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-529235c5-ca5f-47d2-aed6-193a3fad9807 req-c9c84eb0-cd1b-48de-b57e-34889b811a86 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:19.967882 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-529235c5-ca5f-47d2-aed6-193a3fad9807 req-c9c84eb0-cd1b-48de-b57e-34889b811a86 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 944 time: 0.3983364 May 20 04:22:20.007270 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2a99f088-4895-4230-8e77-b324e3184f76 req-8cb3f5bc-7b97-4a58-b852-e31c4970127c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:20.015751 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2a99f088-4895-4230-8e77-b324e3184f76 req-8cb3f5bc-7b97-4a58-b852-e31c4970127c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 932 time: 0.4470353 May 20 04:22:20.095542 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-529235c5-ca5f-47d2-aed6-193a3fad9807 req-8ba7c949-c0c1-49d8-9bd0-e93011642682 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1266992 May 20 04:22:20.140684 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2a99f088-4895-4230-8e77-b324e3184f76 req-798b9f37-e46b-4274-bee1-d75ea666df87 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1236355 May 20 04:22:20.178177 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-034f1eee-144b-49ca-af9a-0682bc622247 req-d94e42ed-ab74-4ad3-83de-93c39cee9bef service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:20.179805 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-034f1eee-144b-49ca-af9a-0682bc622247 req-d94e42ed-ab74-4ad3-83de-93c39cee9bef service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=64bcea8a-6d35-406b-a823-fa1185008174&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1616991 May 20 04:22:20.441610 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-034f1eee-144b-49ca-af9a-0682bc622247 req-cdce430f-37a4-4a93-bf84-8b5beb7409ae service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1937702 May 20 04:22:20.487650 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id a8b0c56a-171d-40e9-beb7-c7b63fa031b9 {{(pid=91644) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:22:20.499394 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:20.515041 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Transition to ACTIVE for port object a8b0c56a-171d-40e9-beb7-c7b63fa031b9 will not be triggered until provisioned by entity L2. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:22:20.537724 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:20.555906 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:20.742147 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-034f1eee-144b-49ca-af9a-0682bc622247 req-1f2f531b-21cd-4832-b9f7-59ef8777efd4 service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2911975 May 20 04:22:20.768923 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-46483198-6ea5-4f34-9808-30418b65da45 req-0c481e42-69a0-47c2-ba4f-6cdac355c523 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:20.770496 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-46483198-6ea5-4f34-9808-30418b65da45 req-0c481e42-69a0-47c2-ba4f-6cdac355c523 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.2856188 May 20 04:22:20.780092 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fe3e9a9e-19ca-4d3e-8192-fd873e85b219 req-b181686a-b09b-48e1-aa56-574f26c3c040 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:20.781661 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fe3e9a9e-19ca-4d3e-8192-fd873e85b219 req-b181686a-b09b-48e1-aa56-574f26c3c040 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 943 time: 0.3005266 May 20 04:22:20.903158 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Pushing event updated for resources: {'Port': ['ID=a8b0c56a-171d-40e9-beb7-c7b63fa031b9,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:20.912684 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-46483198-6ea5-4f34-9808-30418b65da45 req-a56c6fa2-eb4f-44a9-8dc5-f18daaf2822c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1300306 May 20 04:22:20.926544 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fe3e9a9e-19ca-4d3e-8192-fd873e85b219 req-2fd924af-cf29-4434-8d06-778761be23df tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1302297 May 20 04:22:20.966749 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.411s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:20.978508 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-85866db4-dd6e-48ba-96f9-8a5f8c3917e7 req-2ba59b0e-3c79-4ab1-b73a-8aa56fcfbebd service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:20.979997 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-85866db4-dd6e-48ba-96f9-8a5f8c3917e7 req-2ba59b0e-3c79-4ab1-b73a-8aa56fcfbebd service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=9fafdf3a29024ac588e5b24719735677&device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 1096 time: 0.1362314 May 20 04:22:21.101204 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Attempting to bind port a8b0c56a-171d-40e9-beb7-c7b63fa031b9 on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:22:21.101762 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Attempting to bind port a8b0c56a-171d-40e9-beb7-c7b63fa031b9 by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': '0032d9a3-07bf-489f-bb9d-03ab3ff14f98', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 813, 'network_id': '36b37213-8439-447d-9eba-a78baf7c698e'}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:22:21.102089 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Attempting to bind port a8b0c56a-171d-40e9-beb7-c7b63fa031b9 on network 36b37213-8439-447d-9eba-a78baf7c698e {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:22:21.128643 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Port a8b0c56a-171d-40e9-beb7-c7b63fa031b9 on network 36b37213-8439-447d-9eba-a78baf7c698e not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:22:21.128965 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Attempting to bind port a8b0c56a-171d-40e9-beb7-c7b63fa031b9 on network 36b37213-8439-447d-9eba-a78baf7c698e {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:22:21.163532 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 22, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:22:21.163871 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Checking segment: {'id': '0032d9a3-07bf-489f-bb9d-03ab3ff14f98', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 813, 'network_id': '36b37213-8439-447d-9eba-a78baf7c698e'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:22:21.164309 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Bound using segment: {'id': '0032d9a3-07bf-489f-bb9d-03ab3ff14f98', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 813, 'network_id': '36b37213-8439-447d-9eba-a78baf7c698e'} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:22:21.164690 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Bound port: a8b0c56a-171d-40e9-beb7-c7b63fa031b9, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '0032d9a3-07bf-489f-bb9d-03ab3ff14f98', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 813, 'network_id': '36b37213-8439-447d-9eba-a78baf7c698e'}}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:22:21.165030 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:21.183372 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-85866db4-dd6e-48ba-96f9-8a5f8c3917e7 req-258991b7-4333-4138-861a-1d018ff87588 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:21.185601 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-85866db4-dd6e-48ba-96f9-8a5f8c3917e7 req-258991b7-4333-4138-861a-1d018ff87588 service neutron] 10.222.0.74 "GET /v2.0/networks?id=64bcea8a-6d35-406b-a823-fa1185008174 HTTP/1.1" status: 200 len: 918 time: 0.1979210 May 20 04:22:21.198205 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 706277c6-cd8c-4146-ab91-ed9504446690 {{(pid=91644) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:22:21.230451 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:21.266997 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Transition to ACTIVE for port object 706277c6-cd8c-4146-ab91-ed9504446690 will not be triggered until provisioned by entity L2. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:22:21.286261 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:21.298561 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-1d1804cc-6732-4c5f-8995-aaa42d5e5265 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:21.432542 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] For port a8b0c56a-171d-40e9-beb7-c7b63fa031b9, host hv-cinder-78240, cleared binding levels {{(pid=91644) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:22:21.452758 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-85866db4-dd6e-48ba-96f9-8a5f8c3917e7 req-0a3ab40c-6247-430b-b86b-9af8a336dfa2 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=535d5637-ae66-43cb-bd16-3543bd3b7a8b HTTP/1.1" status: 200 len: 217 time: 0.2598972 May 20 04:22:21.551439 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] For port a8b0c56a-171d-40e9-beb7-c7b63fa031b9, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=a8b0c56a-171d-40e9-beb7-c7b63fa031b9,segment=NetworkSegment(0032d9a3-07bf-489f-bb9d-03ab3ff14f98),segment_id=0032d9a3-07bf-489f-bb9d-03ab3ff14f98)] {{(pid=91644) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:22:21.552558 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Pushing event updated for resources: {'Port': ['ID=706277c6-cd8c-4146-ab91-ed9504446690,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:21.563196 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.264s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:21.569595 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:21.583433 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-a877c026-2849-4546-84c2-b96c8bbad20f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:21.626014 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-85866db4-dd6e-48ba-96f9-8a5f8c3917e7 req-a6053fdd-3917-47a9-b5b2-b776f148f2c9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:21.627497 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-85866db4-dd6e-48ba-96f9-8a5f8c3917e7 req-a6053fdd-3917-47a9-b5b2-b776f148f2c9 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=203a3962-2282-4f51-ac92-c9695b5c0cfe HTTP/1.1" status: 200 len: 856 time: 0.1663570 May 20 04:22:21.639387 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:21.640156 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:21.641813 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] 10.222.0.74 "PUT /v2.0/ports/a8b0c56a-171d-40e9-beb7-c7b63fa031b9 HTTP/1.1" status: 200 len: 1094 time: 2.2665961 May 20 04:22:21.650024 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '0d234180-40a1-4229-990e-f796674056ce', 'tag': 'a8b0c56a-171d-40e9-beb7-c7b63fa031b9'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:22:21.654109 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Attempting to bind port 706277c6-cd8c-4146-ab91-ed9504446690 on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:22:21.654463 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Attempting to bind port 706277c6-cd8c-4146-ab91-ed9504446690 by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': '0032d9a3-07bf-489f-bb9d-03ab3ff14f98', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 813, 'network_id': '36b37213-8439-447d-9eba-a78baf7c698e'}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:22:21.654907 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Attempting to bind port 706277c6-cd8c-4146-ab91-ed9504446690 on network 36b37213-8439-447d-9eba-a78baf7c698e {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:22:21.670876 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Port 706277c6-cd8c-4146-ab91-ed9504446690 on network 36b37213-8439-447d-9eba-a78baf7c698e not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:22:21.671310 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Attempting to bind port 706277c6-cd8c-4146-ab91-ed9504446690 on network 36b37213-8439-447d-9eba-a78baf7c698e {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:22:21.683737 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 22, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:22:21.684292 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Checking segment: {'id': '0032d9a3-07bf-489f-bb9d-03ab3ff14f98', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 813, 'network_id': '36b37213-8439-447d-9eba-a78baf7c698e'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:22:21.684755 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Bound using segment: {'id': '0032d9a3-07bf-489f-bb9d-03ab3ff14f98', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 813, 'network_id': '36b37213-8439-447d-9eba-a78baf7c698e'} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:22:21.685279 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Bound port: 706277c6-cd8c-4146-ab91-ed9504446690, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '0032d9a3-07bf-489f-bb9d-03ab3ff14f98', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 813, 'network_id': '36b37213-8439-447d-9eba-a78baf7c698e'}}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:22:21.687202 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:21.689212 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-01514ebe-f741-491c-b465-adbf836a6ca6 req-3aa6d49d-ffde-4a55-b227-ba32284f5e41 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:21.692810 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-01514ebe-f741-491c-b465-adbf836a6ca6 req-3aa6d49d-ffde-4a55-b227-ba32284f5e41 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 944 time: 0.3711910 May 20 04:22:21.737019 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9b3d6623-d645-4b2f-98ad-20fa17972ea4 req-8e7a645f-fb77-41de-9c55-8d7fd542220c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:21.738459 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9b3d6623-d645-4b2f-98ad-20fa17972ea4 req-8e7a645f-fb77-41de-9c55-8d7fd542220c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.3239069 May 20 04:22:21.790612 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-01514ebe-f741-491c-b465-adbf836a6ca6 req-8b4dfbe8-b8cc-4339-8800-6122fbd5a823 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0899687 May 20 04:22:21.813493 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9b3d6623-d645-4b2f-98ad-20fa17972ea4 req-28aa64f4-1de2-41ae-9241-ac0275b9dc2d tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0626607 May 20 04:22:21.826824 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-85866db4-dd6e-48ba-96f9-8a5f8c3917e7 req-ea8dfba1-8a41-48e9-9ac5-fc5ea05ada80 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:21.826824 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-85866db4-dd6e-48ba-96f9-8a5f8c3917e7 req-ea8dfba1-8a41-48e9-9ac5-fc5ea05ada80 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=64bcea8a-6d35-406b-a823-fa1185008174&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1856167 May 20 04:22:21.865342 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] For port 706277c6-cd8c-4146-ab91-ed9504446690, host hv-cinder-78240, cleared binding levels {{(pid=91644) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:22:21.882426 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Pushing event updated for resources: {'Port': ['ID=a8b0c56a-171d-40e9-beb7-c7b63fa031b9,revision_number=3']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:21.892673 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] For port 706277c6-cd8c-4146-ab91-ed9504446690, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=706277c6-cd8c-4146-ab91-ed9504446690,segment=NetworkSegment(0032d9a3-07bf-489f-bb9d-03ab3ff14f98),segment_id=0032d9a3-07bf-489f-bb9d-03ab3ff14f98)] {{(pid=91644) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:22:21.896916 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.314s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:21.898605 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '0d234180-40a1-4229-990e-f796674056ce', 'tag': 'a8b0c56a-171d-40e9-beb7-c7b63fa031b9', 'status': 'completed', 'code': 200} May 20 04:22:21.905365 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:21.916952 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:21.973274 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:21.973947 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:21.975263 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] 10.222.0.74 "PUT /v2.0/ports/706277c6-cd8c-4146-ab91-ed9504446690 HTTP/1.1" status: 200 len: 1093 time: 2.6666923 May 20 04:22:22.007790 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-8174ddd3-3f77-431c-8d45-2238fd5cd9e5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:22.009256 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-8174ddd3-3f77-431c-8d45-2238fd5cd9e5 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=bec56d8703cf47f3b7193b7d1f55396d&device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 1097 time: 0.2958739 May 20 04:22:22.018619 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-85866db4-dd6e-48ba-96f9-8a5f8c3917e7 req-b20c4141-8fcd-4e2f-b5f2-0157a5183fae service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1861613 May 20 04:22:22.019524 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Pushing event updated for resources: {'Port': ['ID=706277c6-cd8c-4146-ab91-ed9504446690,revision_number=3']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:22.022684 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.106s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:22.148316 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-0ee62d49-7a7e-461e-af62-057db5ab89d6 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.13&port_id=a8b0c56a-171d-40e9-beb7-c7b63fa031b9 HTTP/1.1" status: 200 len: 217 time: 0.1207678 May 20 04:22:22.235015 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-d865fb8c-71ff-43ab-a943-d642f7126923 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:22.236463 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-d865fb8c-71ff-43ab-a943-d642f7126923 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=b25ae9c8-e11d-45ee-bde2-332cc1538602 HTTP/1.1" status: 200 len: 859 time: 0.0767400 May 20 04:22:22.242910 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-dbdf6e66-823d-4323-a48c-8ef38098460c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:22.244673 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-dbdf6e66-823d-4323-a48c-8ef38098460c service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=bec56d8703cf47f3b7193b7d1f55396d&device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 1096 time: 0.1493351 May 20 04:22:22.346379 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-7e2abb7e-43b5-402c-a216-7192d8944447 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=36b37213-8439-447d-9eba-a78baf7c698e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0990198 May 20 04:22:22.347728 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-b5551c08-d104-4b1e-97e1-b4487801bd17 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=706277c6-cd8c-4146-ab91-ed9504446690 HTTP/1.1" status: 200 len: 217 time: 0.0714169 May 20 04:22:22.372888 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-85866db4-dd6e-48ba-96f9-8a5f8c3917e7 req-344c9ffd-50e3-4b83-9540-fb8c5a15c0fc service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3399246 May 20 04:22:22.384500 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4226449e-866b-4215-8496-4a053f250468 req-ea3fea65-8154-40c7-889e-f00c339b2ed9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:22.387244 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4226449e-866b-4215-8496-4a053f250468 req-ea3fea65-8154-40c7-889e-f00c339b2ed9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.3324420 May 20 04:22:22.412291 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5c9d13c2-7c47-4183-b885-5df20dd08302 req-67bcbff3-504c-4d46-8929-27db7036cef9 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:22.413786 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5c9d13c2-7c47-4183-b885-5df20dd08302 req-67bcbff3-504c-4d46-8929-27db7036cef9 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 943 time: 0.3064201 May 20 04:22:22.469316 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4226449e-866b-4215-8496-4a053f250468 req-03adbeed-9962-4f06-8fa2-17975a8ddb41 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0772882 May 20 04:22:22.477352 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5c9d13c2-7c47-4183-b885-5df20dd08302 req-5136896a-5015-432f-852e-186b0b5a10ab tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0557396 May 20 04:22:22.580222 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-e447738f-4353-4b74-896e-16081c27663b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:22.581718 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-e447738f-4353-4b74-896e-16081c27663b service neutron] 10.222.0.74 "GET /v2.0/subnets?id=b25ae9c8-e11d-45ee-bde2-332cc1538602 HTTP/1.1" status: 200 len: 859 time: 0.2015443 May 20 04:22:22.643071 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-297a5f7f-d8a6-48bd-ac94-487b559450cd service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=36b37213-8439-447d-9eba-a78baf7c698e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0537562 May 20 04:22:22.648230 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-15804274-869a-4c61-a22d-9271da4c21e1 service neutron] 10.222.0.74 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2866726 May 20 04:22:22.963095 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-99da97ab-c410-4169-8096-778dd2b3e231 service neutron] 10.222.0.74 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3031211 May 20 04:22:22.964944 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c80aeaac-ea48-4d72-aad4-a8e7b6332396 req-5aba6077-64a0-4c70-b9ff-542ac2f11e1d service neutron] 10.222.0.74 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3037012 May 20 04:22:23.116584 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-118aeb95-6339-482a-907c-e4294c671f8e req-36cc9e52-be97-409c-b823-e3c3fd5e06cb tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:23.118032 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-118aeb95-6339-482a-907c-e4294c671f8e req-36cc9e52-be97-409c-b823-e3c3fd5e06cb tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 944 time: 0.1891363 May 20 04:22:23.131086 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1ac9c1a1-6a8a-49d7-94f2-f87d395347de req-0b21d87a-87d7-4979-aa8c-2d1a29348f4e tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:23.132686 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1ac9c1a1-6a8a-49d7-94f2-f87d395347de req-0b21d87a-87d7-4979-aa8c-2d1a29348f4e tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.1304500 May 20 04:22:23.188002 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b19a4a1b-fd4d-4054-b41c-db2470b70d5b req-89f3a825-ccc0-4fa3-a363-003229bfb9fb service neutron] 10.222.0.74 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2118115 May 20 04:22:23.212897 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-19d67faf-d8d2-413b-8cae-3fc00b435f75 req-eac75a91-0084-4f09-a08b-86fbc2f8a2da service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:23.215679 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-19d67faf-d8d2-413b-8cae-3fc00b435f75 req-eac75a91-0084-4f09-a08b-86fbc2f8a2da service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=bec56d8703cf47f3b7193b7d1f55396d&device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 1097 time: 0.1807325 May 20 04:22:23.222618 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-118aeb95-6339-482a-907c-e4294c671f8e req-b99956ba-d18d-414f-9fee-1ea5939c4c49 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0889051 May 20 04:22:23.228047 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1ac9c1a1-6a8a-49d7-94f2-f87d395347de req-481292b6-e8eb-42cf-8757-ddb8e0b0e47f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0805552 May 20 04:22:23.368625 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-19d67faf-d8d2-413b-8cae-3fc00b435f75 req-441fda86-6ebf-473a-9583-5691511cb7ad service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:23.370059 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-19d67faf-d8d2-413b-8cae-3fc00b435f75 req-441fda86-6ebf-473a-9583-5691511cb7ad service neutron] 10.222.0.74 "GET /v2.0/networks?id=36b37213-8439-447d-9eba-a78baf7c698e HTTP/1.1" status: 200 len: 920 time: 0.1372128 May 20 04:22:23.474442 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 34076) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:23.610167 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a35e4b54-eabe-4bc7-98c0-9d88b420d72b admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:23.624242 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-a35e4b54-eabe-4bc7-98c0-9d88b420d72b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:22:23.628060 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-a35e4b54-eabe-4bc7-98c0-9d88b420d72b admin admin] Usage tracker for resource:security_group_rule and tenant:180d1a447e99408f9c9d9548a0520c3a is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:22:23.630747 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-a35e4b54-eabe-4bc7-98c0-9d88b420d72b admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:22:23.636911 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-a35e4b54-eabe-4bc7-98c0-9d88b420d72b admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:22:23.719378 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a35e4b54-eabe-4bc7-98c0-9d88b420d72b admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:23.724488 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a35e4b54-eabe-4bc7-98c0-9d88b420d72b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:23.733836 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-53529cea-79d7-49fd-8012-eb5cd87eeef1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:23.734309 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f725fe69-d3f4-44ee-8898-f4c9a67a3896 req-9e7e80e7-9e9e-44e3-8c6a-af211774775f tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:23.735844 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f725fe69-d3f4-44ee-8898-f4c9a67a3896 req-9e7e80e7-9e9e-44e3-8c6a-af211774775f tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 943 time: 0.1523731 May 20 04:22:23.747324 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-aa426ffc-0c79-4f6b-9949-a891f3ed5689 req-20433d04-b97c-4526-9ab2-7edf255ca955 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:23.748719 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-aa426ffc-0c79-4f6b-9949-a891f3ed5689 req-20433d04-b97c-4526-9ab2-7edf255ca955 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.1262133 May 20 04:22:23.776165 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-a35e4b54-eabe-4bc7-98c0-9d88b420d72b admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=180d1a447e99408f9c9d9548a0520c3a&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3006239 May 20 04:22:23.777760 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a35e4b54-eabe-4bc7-98c0-9d88b420d72b admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=adec8cff-122d-4084-86aa-23716ad113b9,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:23.780015 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 34086) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:23.782558 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-a35e4b54-eabe-4bc7-98c0-9d88b420d72b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:23.823739 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-aa426ffc-0c79-4f6b-9949-a891f3ed5689 req-43f5484f-0437-4e17-a513-961b6ead6976 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0610726 May 20 04:22:23.826949 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f725fe69-d3f4-44ee-8898-f4c9a67a3896 req-84a25bcb-a7a7-4b78-bac0-f33f39ba026a tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0773444 May 20 04:22:23.844397 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b1e4a4a8-2e70-4802-8d88-9544e2c048d2 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:23.869497 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b1e4a4a8-2e70-4802-8d88-9544e2c048d2 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:23.900077 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'bbf6ae7d-8ff5-4b69-adef-47220d6c5c93', 'tag': '706277c6-cd8c-4146-ab91-ed9504446690'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:22:23.928510 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b1e4a4a8-2e70-4802-8d88-9544e2c048d2 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:23.933492 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-59b2047d-8f1f-4db1-b8e4-90f31d340654 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:23.944062 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b1e4a4a8-2e70-4802-8d88-9544e2c048d2 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=adec8cff-122d-4084-86aa-23716ad113b9,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:23.947568 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-b1e4a4a8-2e70-4802-8d88-9544e2c048d2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.014s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:23.950157 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'bbf6ae7d-8ff5-4b69-adef-47220d6c5c93', 'tag': '706277c6-cd8c-4146-ab91-ed9504446690', 'status': 'completed', 'code': 200} May 20 04:22:23.952065 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b1e4a4a8-2e70-4802-8d88-9544e2c048d2 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:23.953126 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-b1e4a4a8-2e70-4802-8d88-9544e2c048d2 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/adec8cff-122d-4084-86aa-23716ad113b9 HTTP/1.1" status: 204 len: 168 time: 0.1722534 May 20 04:22:24.270752 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 34104) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:24.289654 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d22f1933-748d-4089-8fb3-332ddc6447c2 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:24.304003 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-d22f1933-748d-4089-8fb3-332ddc6447c2 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:22:24.309620 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-d22f1933-748d-4089-8fb3-332ddc6447c2 admin admin] Usage tracker for resource:security_group_rule and tenant:5bd0610f65e94b8296ecc1269b7b0aa4 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:22:24.312536 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-d22f1933-748d-4089-8fb3-332ddc6447c2 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:22:24.318179 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-d22f1933-748d-4089-8fb3-332ddc6447c2 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:22:24.411371 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d22f1933-748d-4089-8fb3-332ddc6447c2 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:24.417042 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d22f1933-748d-4089-8fb3-332ddc6447c2 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:24.427707 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-fcfa7b9f-3c2a-4aa6-ac1d-869ec8bdd4b4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:24.468689 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d22f1933-748d-4089-8fb3-332ddc6447c2 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=1b3dea0c-3bae-418e-ab2a-6650eb08db7b,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:24.475734 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-d22f1933-748d-4089-8fb3-332ddc6447c2 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=5bd0610f65e94b8296ecc1269b7b0aa4&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2037251 May 20 04:22:24.476641 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d22f1933-748d-4089-8fb3-332ddc6447c2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:24.477113 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 34110) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:24.480159 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3a3bb1c1-1a33-4de5-b74d-1864db8121e1 req-0182e932-e91b-4cda-97b0-3382b5d1cbca tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:24.487926 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3a3bb1c1-1a33-4de5-b74d-1864db8121e1 req-0182e932-e91b-4cda-97b0-3382b5d1cbca tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.1471047 May 20 04:22:24.537656 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c6762810-8206-4694-8b4f-7c452b292de0 req-b9ceb945-adad-4f26-822c-e0b4529cfd5e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:24.538137 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9d7e2ca2-0be3-4d4c-942d-e86b1d4133e0 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:24.539156 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c6762810-8206-4694-8b4f-7c452b292de0 req-b9ceb945-adad-4f26-822c-e0b4529cfd5e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 944 time: 0.1529148 May 20 04:22:24.546766 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3a3bb1c1-1a33-4de5-b74d-1864db8121e1 req-c0399017-c568-40cc-b90f-2ef1a2a1feb2 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0579340 May 20 04:22:24.569410 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9d7e2ca2-0be3-4d4c-942d-e86b1d4133e0 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:24.588994 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c6762810-8206-4694-8b4f-7c452b292de0 req-de93be26-c881-4615-8410-66709c8fa2d1 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0412064 May 20 04:22:24.631906 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9d7e2ca2-0be3-4d4c-942d-e86b1d4133e0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:24.635557 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:24.646340 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9d7e2ca2-0be3-4d4c-942d-e86b1d4133e0 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=1b3dea0c-3bae-418e-ab2a-6650eb08db7b,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:24.649813 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-9d7e2ca2-0be3-4d4c-942d-e86b1d4133e0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.014s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:24.653208 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9d7e2ca2-0be3-4d4c-942d-e86b1d4133e0 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:24.654274 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-9d7e2ca2-0be3-4d4c-942d-e86b1d4133e0 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/1b3dea0c-3bae-418e-ab2a-6650eb08db7b HTTP/1.1" status: 204 len: 168 time: 0.1763141 May 20 04:22:25.086247 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e9d97745-feab-447b-8c21-e7bd6bd00d50 req-e771f3e4-a697-40e3-a61e-f7be4762612d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:25.092805 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e9d97745-feab-447b-8c21-e7bd6bd00d50 req-e771f3e4-a697-40e3-a61e-f7be4762612d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.1531177 May 20 04:22:25.108904 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0c8843d4-b928-4e87-a2bd-69c5d8c668b8 req-dd49d6e4-a5cb-40da-b190-50215cd6f0f4 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:25.110441 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0c8843d4-b928-4e87-a2bd-69c5d8c668b8 req-dd49d6e4-a5cb-40da-b190-50215cd6f0f4 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 943 time: 0.1340261 May 20 04:22:25.177291 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e9d97745-feab-447b-8c21-e7bd6bd00d50 req-48ab763d-51af-4fe7-b0c3-cf606add467a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0658374 May 20 04:22:25.184155 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0c8843d4-b928-4e87-a2bd-69c5d8c668b8 req-107a25ef-e984-4c7d-a163-3be763dc810c tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0576658 May 20 04:22:25.787838 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0eb9ea40-6e0c-450c-9382-0c40b461897e req-954224da-ef47-4320-aea1-50214c899d04 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:25.789306 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0eb9ea40-6e0c-450c-9382-0c40b461897e req-954224da-ef47-4320-aea1-50214c899d04 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.1369636 May 20 04:22:25.834757 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-71877bbb-2feb-4af5-a1ef-6297cce8bfd4 req-ba8c6e8d-26c1-4148-a6cb-dd14ac666790 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:25.838109 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-71877bbb-2feb-4af5-a1ef-6297cce8bfd4 req-ba8c6e8d-26c1-4148-a6cb-dd14ac666790 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 944 time: 0.1292188 May 20 04:22:25.842829 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device efb81444-f373-4e9c-a0aa-e3270bd4b980 details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:22:25.875165 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0eb9ea40-6e0c-450c-9382-0c40b461897e req-d156f79c-3197-4d0f-bd33-6c2b22ab37c8 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0791411 May 20 04:22:25.897792 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-71877bbb-2feb-4af5-a1ef-6297cce8bfd4 req-780cc284-3fc2-4e08-bc06-419d68a6fd08 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0493343 May 20 04:22:25.920478 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-19d67faf-d8d2-413b-8cae-3fc00b435f75 req-ce06980e-1b8c-477e-9380-572e258fc23e service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.13&port_id=a8b0c56a-171d-40e9-beb7-c7b63fa031b9 HTTP/1.1" status: 200 len: 217 time: 0.1014745 May 20 04:22:26.009117 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port efb81444-f373-4e9c-a0aa-e3270bd4b980, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=efb81444-f373-4e9c-a0aa-e3270bd4b980,segment=NetworkSegment(3c4a2238-8a32-4d92-a1b1-8af91c1c27a7),segment_id=3c4a2238-8a32-4d92-a1b1-8af91c1c27a7)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:22:26.021954 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': 'efb81444-f373-4e9c-a0aa-e3270bd4b980', 'network_id': 'a009b619-8ab3-40e3-b5f3-9cc97ccd3415', 'port_id': 'efb81444-f373-4e9c-a0aa-e3270bd4b980', 'mac_address': 'fa:16:3e:5b:8f:11', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 814, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'b9f2dbbb-b724-44b9-878b-2267c4d95aee', 'ip_address': '172.20.0.2'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:22:26.078507 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:26.098347 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id efb81444-f373-4e9c-a0aa-e3270bd4b980 {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:22:26.116202 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port efb81444-f373-4e9c-a0aa-e3270bd4b980, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=efb81444-f373-4e9c-a0aa-e3270bd4b980,segment=NetworkSegment(3c4a2238-8a32-4d92-a1b1-8af91c1c27a7),segment_id=3c4a2238-8a32-4d92-a1b1-8af91c1c27a7)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:22:26.122328 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:26.123134 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 849868be-af51-4967-9a6c-93d23cecb450 details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:22:26.125752 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:26.196035 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=efb81444-f373-4e9c-a0aa-e3270bd4b980,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:26.199251 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:26.335362 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 849868be-af51-4967-9a6c-93d23cecb450, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=849868be-af51-4967-9a6c-93d23cecb450,segment=NetworkSegment(6fe99325-a72c-48bf-abc7-9bbcc4b753ff),segment_id=6fe99325-a72c-48bf-abc7-9bbcc4b753ff)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:22:26.348734 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': '849868be-af51-4967-9a6c-93d23cecb450', 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f', 'port_id': '849868be-af51-4967-9a6c-93d23cecb450', 'mac_address': 'fa:16:3e:26:ce:70', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 836, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '0f918085-8d33-4e36-a59e-7f270a2b1874', 'ip_address': '172.20.0.14'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:22:26.402219 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:26.418626 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 849868be-af51-4967-9a6c-93d23cecb450 {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:22:26.434363 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 849868be-af51-4967-9a6c-93d23cecb450, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=849868be-af51-4967-9a6c-93d23cecb450,segment=NetworkSegment(6fe99325-a72c-48bf-abc7-9bbcc4b753ff),segment_id=6fe99325-a72c-48bf-abc7-9bbcc4b753ff)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:22:26.439027 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:26.441973 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:26.445460 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1a80c046-85b7-42d4-b3ed-1df52e68b05d req-880cc4c2-73ea-4373-907e-e6a49e446300 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:26.446923 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1a80c046-85b7-42d4-b3ed-1df52e68b05d req-880cc4c2-73ea-4373-907e-e6a49e446300 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 944 time: 0.1385636 May 20 04:22:26.455010 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-60c6c4ee-68d0-4fc3-b912-843404cdaf77 req-e7874fd6-1e34-43f6-999c-62b666533f3d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:26.456392 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-60c6c4ee-68d0-4fc3-b912-843404cdaf77 req-e7874fd6-1e34-43f6-999c-62b666533f3d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.1502283 May 20 04:22:26.535195 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=849868be-af51-4967-9a6c-93d23cecb450,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:26.536102 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1a80c046-85b7-42d4-b3ed-1df52e68b05d req-2f2e9f56-72c7-4538-a5c2-9573de2ad7fe tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0787745 May 20 04:22:26.538233 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:26.538857 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-60c6c4ee-68d0-4fc3-b912-843404cdaf77 req-ccda2f49-73c3-467e-951b-7d4d12123b62 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0675385 May 20 04:22:27.129224 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2c370bab-7dd7-4593-890d-9c4e964ebe7d req-de8063a4-feac-46f2-82fd-17811e2e66f2 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:27.130661 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2c370bab-7dd7-4593-890d-9c4e964ebe7d req-de8063a4-feac-46f2-82fd-17811e2e66f2 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.1381500 May 20 04:22:27.141025 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d881e330-bd75-43c5-b0d5-f9f9ecab9532 req-db9f8097-b575-4958-bb2b-794fbe98661f tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:27.142386 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d881e330-bd75-43c5-b0d5-f9f9ecab9532 req-db9f8097-b575-4958-bb2b-794fbe98661f tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 945 time: 0.1143353 May 20 04:22:27.209637 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2c370bab-7dd7-4593-890d-9c4e964ebe7d req-be756167-5584-4728-af84-8682925d0088 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0663280 May 20 04:22:27.214280 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d881e330-bd75-43c5-b0d5-f9f9ecab9532 req-8513dd4b-89b6-4ec4-b891-80435e4e41be tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0582933 May 20 04:22:27.845216 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9654942d-4922-4bdf-94b0-b413b3f450e5 req-55a59655-33fe-4072-9551-6b72a1421c55 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:27.846561 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9654942d-4922-4bdf-94b0-b413b3f450e5 req-55a59655-33fe-4072-9551-6b72a1421c55 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 944 time: 0.1985910 May 20 04:22:27.856864 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d193491c-cf82-4512-a62f-fc37fb97176a req-0ed6c77f-df7a-4631-8a98-ce6b4f7b8766 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:27.858218 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d193491c-cf82-4512-a62f-fc37fb97176a req-0ed6c77f-df7a-4631-8a98-ce6b4f7b8766 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.1160071 May 20 04:22:27.923855 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9654942d-4922-4bdf-94b0-b413b3f450e5 req-81af2cdb-56be-481c-8520-dac34ffb7334 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0646942 May 20 04:22:27.928170 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d193491c-cf82-4512-a62f-fc37fb97176a req-c05e1795-7dd0-4e9e-ba94-e8fdd3cd3b03 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0565336 May 20 04:22:28.455615 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b3574997-4fe0-4c6e-a33d-823eeb3864a8 req-14df1b2c-4aee-4f05-a680-76654cbd73c1 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:28.457008 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b3574997-4fe0-4c6e-a33d-823eeb3864a8 req-14df1b2c-4aee-4f05-a680-76654cbd73c1 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.1400976 May 20 04:22:28.464859 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e76396aa-c118-4e2b-b71d-5e16d64651de req-e318216e-2cbc-428e-a14e-d9458c0931f4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:28.466174 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e76396aa-c118-4e2b-b71d-5e16d64651de req-e318216e-2cbc-428e-a14e-d9458c0931f4 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 945 time: 0.1171434 May 20 04:22:28.532189 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b3574997-4fe0-4c6e-a33d-823eeb3864a8 req-844751d1-bf16-476f-a98c-f3fb1dd14fb2 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0650823 May 20 04:22:28.540407 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e76396aa-c118-4e2b-b71d-5e16d64651de req-65c8425d-65b3-4e18-b606-7879106d44e5 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0597177 May 20 04:22:28.642441 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-19d67faf-d8d2-413b-8cae-3fc00b435f75 req-256159f3-61e5-4577-8d82-86c2dce3b99c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:28.643921 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-19d67faf-d8d2-413b-8cae-3fc00b435f75 req-256159f3-61e5-4577-8d82-86c2dce3b99c service neutron] 10.222.0.74 "GET /v2.0/subnets?id=b25ae9c8-e11d-45ee-bde2-332cc1538602 HTTP/1.1" status: 200 len: 859 time: 0.1193857 May 20 04:22:28.693235 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-19d67faf-d8d2-413b-8cae-3fc00b435f75 req-0b223346-8273-47a3-89e6-0d1bf149b11c service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=36b37213-8439-447d-9eba-a78baf7c698e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0426574 May 20 04:22:28.724385 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3d96dbe4-3f45-43d0-802e-cf13bf88d60b req-c3ca8d6d-9e54-43ba-a4cd-267d20363a34 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:28.726138 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-3d96dbe4-3f45-43d0-802e-cf13bf88d60b req-c3ca8d6d-9e54-43ba-a4cd-267d20363a34 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=bec56d8703cf47f3b7193b7d1f55396d&device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 1096 time: 0.1610184 May 20 04:22:28.992441 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-19d67faf-d8d2-413b-8cae-3fc00b435f75 req-954faa33-0b47-40de-8c55-d58cde998467 service neutron] 10.222.0.74 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2905302 May 20 04:22:28.997212 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3d96dbe4-3f45-43d0-802e-cf13bf88d60b req-fa40cef9-0519-42ad-a4e4-b68d73beb038 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:28.998516 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-3d96dbe4-3f45-43d0-802e-cf13bf88d60b req-fa40cef9-0519-42ad-a4e4-b68d73beb038 service neutron] 10.222.0.74 "GET /v2.0/networks?id=36b37213-8439-447d-9eba-a78baf7c698e HTTP/1.1" status: 200 len: 920 time: 0.2591457 May 20 04:22:29.105052 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-3d96dbe4-3f45-43d0-802e-cf13bf88d60b req-282c5c6c-73cc-4cde-b19d-4ea7ced8a9e2 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=706277c6-cd8c-4146-ab91-ed9504446690 HTTP/1.1" status: 200 len: 217 time: 0.0834739 May 20 04:22:29.181380 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-14048e34-3833-4890-80a2-21c9a26096f0 req-59091e9e-518b-4f2c-a652-0f76757b52d9 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:29.182812 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-14048e34-3833-4890-80a2-21c9a26096f0 req-59091e9e-518b-4f2c-a652-0f76757b52d9 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 944 time: 0.1508815 May 20 04:22:29.190743 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e268933f-8c05-43b9-ac25-be0d0c0b2b02 req-33449092-b070-440a-adfa-efac58f9f8be tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:29.192329 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e268933f-8c05-43b9-ac25-be0d0c0b2b02 req-33449092-b070-440a-adfa-efac58f9f8be tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.1205914 May 20 04:22:29.233665 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3d96dbe4-3f45-43d0-802e-cf13bf88d60b req-bc77c0e1-50a7-4d65-9659-26f2a3a2fa16 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:29.235083 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-3d96dbe4-3f45-43d0-802e-cf13bf88d60b req-bc77c0e1-50a7-4d65-9659-26f2a3a2fa16 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=b25ae9c8-e11d-45ee-bde2-332cc1538602 HTTP/1.1" status: 200 len: 859 time: 0.0845275 May 20 04:22:29.263172 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-14048e34-3833-4890-80a2-21c9a26096f0 req-e844369e-7d85-456f-8872-228f9d5d321d tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0699203 May 20 04:22:29.270523 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e268933f-8c05-43b9-ac25-be0d0c0b2b02 req-200217df-849f-4da8-9d7c-1405e4f16277 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0630758 May 20 04:22:29.274064 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-19d67faf-d8d2-413b-8cae-3fc00b435f75 req-f672e9c7-e86c-44e2-8d60-3b2c15967168 service neutron] 10.222.0.74 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2746301 May 20 04:22:29.295384 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-3d96dbe4-3f45-43d0-802e-cf13bf88d60b req-54b2ae69-8470-4100-a262-fa26db49f5fe service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=36b37213-8439-447d-9eba-a78baf7c698e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0526001 May 20 04:22:29.438863 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-3d96dbe4-3f45-43d0-802e-cf13bf88d60b req-a8c8d2bf-6e73-4fc1-b0d5-5e3dde79ca40 service neutron] 10.222.0.74 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1268258 May 20 04:22:29.579163 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-3d96dbe4-3f45-43d0-802e-cf13bf88d60b req-e44ff328-aba2-4a00-b973-7e23d689dd9a service neutron] 10.222.0.74 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1330035 May 20 04:22:29.785268 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0e8f9e09-9b8f-45cd-a857-bbcb62d3d006 req-bdf6ca53-61d0-4723-9d5a-93c05641b237 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:29.786668 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0e8f9e09-9b8f-45cd-a857-bbcb62d3d006 req-bdf6ca53-61d0-4723-9d5a-93c05641b237 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 945 time: 0.1127999 May 20 04:22:29.794225 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-aaf3b395-115b-4635-8d63-46ecdefe37ea req-dd1c2612-605a-44e6-a1fd-32269b90ae50 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:29.795511 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-aaf3b395-115b-4635-8d63-46ecdefe37ea req-dd1c2612-605a-44e6-a1fd-32269b90ae50 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.1521778 May 20 04:22:29.861110 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0e8f9e09-9b8f-45cd-a857-bbcb62d3d006 req-a17d4133-830c-4659-b1e9-bdb9c3f864bb tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0646567 May 20 04:22:29.868246 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-aaf3b395-115b-4635-8d63-46ecdefe37ea req-11e662f3-fa16-482a-8b20-892a9ed5bbd6 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0573058 May 20 04:22:30.511239 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6e4a7c21-ba74-4ecf-ad11-e642df370bc4 req-842e2350-1cf1-47bf-a78e-f506c12d25cb tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:30.512839 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6e4a7c21-ba74-4ecf-ad11-e642df370bc4 req-842e2350-1cf1-47bf-a78e-f506c12d25cb tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 944 time: 0.1399777 May 20 04:22:30.549121 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bd09b9cd-1c86-48ca-81de-3662cc506adf req-67cd1192-5f2e-4d94-ad8d-260bc9fe1119 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:30.551574 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bd09b9cd-1c86-48ca-81de-3662cc506adf req-67cd1192-5f2e-4d94-ad8d-260bc9fe1119 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.1427319 May 20 04:22:30.591834 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6e4a7c21-ba74-4ecf-ad11-e642df370bc4 req-45c04020-19b4-42b9-afc1-7322f45788f0 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0707891 May 20 04:22:30.629915 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bd09b9cd-1c86-48ca-81de-3662cc506adf req-a19e1e3c-43f2-4bf3-a245-2ddc3cdcb0de tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0667341 May 20 04:22:31.119273 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9f523dac-e8d7-428a-b1d3-7f094d8593ed req-2db67304-e471-424c-8f5e-77dcb928966d tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:31.120723 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9f523dac-e8d7-428a-b1d3-7f094d8593ed req-2db67304-e471-424c-8f5e-77dcb928966d tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 945 time: 0.1393790 May 20 04:22:31.130871 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-119c0a90-cc43-43af-94f0-120ef64124f9 req-cd133004-5220-4ceb-b1aa-31c613d7df13 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:31.132321 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-119c0a90-cc43-43af-94f0-120ef64124f9 req-cd133004-5220-4ceb-b1aa-31c613d7df13 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.1160038 May 20 04:22:31.199685 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9f523dac-e8d7-428a-b1d3-7f094d8593ed req-086d39a8-27fc-4efc-93c6-7b59ca39f8d5 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0662932 May 20 04:22:31.206614 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-119c0a90-cc43-43af-94f0-120ef64124f9 req-37f00069-f935-4b21-99ff-f6e1eac2e77f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0590868 May 20 04:22:31.847357 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-39a084ca-e3ef-4522-b377-6b7fe6a2571e req-b3134703-8e7d-43c3-a458-863e711bc04c tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:31.848769 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-39a084ca-e3ef-4522-b377-6b7fe6a2571e req-b3134703-8e7d-43c3-a458-863e711bc04c tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 944 time: 0.1398208 May 20 04:22:31.858767 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1f2c6cd3-7307-4a1d-9246-714aaa2afabb req-18ff32df-e167-40bb-955c-f7c4ae8057d5 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:31.860125 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1f2c6cd3-7307-4a1d-9246-714aaa2afabb req-18ff32df-e167-40bb-955c-f7c4ae8057d5 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.1157827 May 20 04:22:31.927881 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-39a084ca-e3ef-4522-b377-6b7fe6a2571e req-74b0bd2f-73e5-437b-8efc-f33858652c59 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0667512 May 20 04:22:31.932163 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1f2c6cd3-7307-4a1d-9246-714aaa2afabb req-d1d9b1f2-6e42-468c-aa9e-c31013f41dc3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0581658 May 20 04:22:32.331547 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 535d5637-ae66-43cb-bd16-3543bd3b7a8b details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:22:32.456773 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-891552b9-b21d-486f-9deb-ff00d6b32a2d req-de3f0adc-5a0d-45e9-a704-0f4e8dab7f0e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:32.458270 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-891552b9-b21d-486f-9deb-ff00d6b32a2d req-de3f0adc-5a0d-45e9-a704-0f4e8dab7f0e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 945 time: 0.1390488 May 20 04:22:32.470398 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 535d5637-ae66-43cb-bd16-3543bd3b7a8b, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=535d5637-ae66-43cb-bd16-3543bd3b7a8b,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:22:32.484591 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': '535d5637-ae66-43cb-bd16-3543bd3b7a8b', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'port_id': '535d5637-ae66-43cb-bd16-3543bd3b7a8b', 'mac_address': 'fa:16:3e:ea:90:d5', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 847, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.6'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:22:32.485517 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7c95595d-d0cf-4fbc-b9dd-db20aa8737ad req-9be665d3-6a2a-4f46-95d1-0f3ac8902958 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:32.486952 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7c95595d-d0cf-4fbc-b9dd-db20aa8737ad req-9be665d3-6a2a-4f46-95d1-0f3ac8902958 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.1324449 May 20 04:22:32.539858 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:32.540206 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-891552b9-b21d-486f-9deb-ff00d6b32a2d req-a7c62ae1-fce7-4300-8d58-8d6643c1f007 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0755620 May 20 04:22:32.551417 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7c95595d-d0cf-4fbc-b9dd-db20aa8737ad req-0f8f9a69-50de-4076-b4dd-2678b6be5f46 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0583057 May 20 04:22:32.561285 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 535d5637-ae66-43cb-bd16-3543bd3b7a8b {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:22:32.578036 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 535d5637-ae66-43cb-bd16-3543bd3b7a8b, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=535d5637-ae66-43cb-bd16-3543bd3b7a8b,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:22:32.582556 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:32.584198 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:32.651772 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=535d5637-ae66-43cb-bd16-3543bd3b7a8b,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:32.654778 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.070s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:33.253029 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c703015c-6777-42bd-b553-3bf22111ab9b req-69fc14c7-a6cc-4576-8b65-ff0a9fad985d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:33.254455 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c703015c-6777-42bd-b553-3bf22111ab9b req-69fc14c7-a6cc-4576-8b65-ff0a9fad985d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 944 time: 0.2055006 May 20 04:22:33.261637 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6b0fcd36-b6ee-4116-97c6-7b84b4062797 req-8f628b99-f5f0-4973-a8cc-a628eb95c2d6 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:33.262937 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6b0fcd36-b6ee-4116-97c6-7b84b4062797 req-8f628b99-f5f0-4973-a8cc-a628eb95c2d6 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 944 time: 0.2120895 May 20 04:22:33.329562 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c703015c-6777-42bd-b553-3bf22111ab9b req-32ef645f-e593-428e-ab9d-323cc90bf64b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0656459 May 20 04:22:33.336794 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6b0fcd36-b6ee-4116-97c6-7b84b4062797 req-d789cca8-acc7-4a42-ad82-147d481cc378 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0600576 May 20 04:22:33.809764 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-20d3e854-452c-4991-b3c3-ff4b2039eecf req-616f0538-e43a-448a-8bb1-0e2e7d887c37 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:33.811311 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-20d3e854-452c-4991-b3c3-ff4b2039eecf req-616f0538-e43a-448a-8bb1-0e2e7d887c37 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.1434243 May 20 04:22:33.818956 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-49c257ae-f8e9-4457-bd23-e82dbf3db3bb req-09a68e3c-db15-4e12-9004-bba91bda33f5 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:33.820328 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-49c257ae-f8e9-4457-bd23-e82dbf3db3bb req-09a68e3c-db15-4e12-9004-bba91bda33f5 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 945 time: 0.1512799 May 20 04:22:33.886177 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-20d3e854-452c-4991-b3c3-ff4b2039eecf req-30790ec9-76f5-441b-9a60-fa8cbce36e61 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0649552 May 20 04:22:33.893055 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-49c257ae-f8e9-4457-bd23-e82dbf3db3bb req-75f5b110-5c28-4b5b-a2cd-5f3c6cb771fe tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0574470 May 20 04:22:34.595576 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-656cc96f-db8e-41d9-be99-7e47a1723051 req-cf84c330-1b10-4b04-8337-23d912d6845d tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:34.597011 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-656cc96f-db8e-41d9-be99-7e47a1723051 req-cf84c330-1b10-4b04-8337-23d912d6845d tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 944 time: 0.1419568 May 20 04:22:34.605531 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3350a7ce-7866-402b-bd03-10f2912846b6 req-489858b6-7f27-4049-ac20-5f89f31e75ab tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:34.607305 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3350a7ce-7866-402b-bd03-10f2912846b6 req-489858b6-7f27-4049-ac20-5f89f31e75ab tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 944 time: 0.1208951 May 20 04:22:34.675522 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-656cc96f-db8e-41d9-be99-7e47a1723051 req-2bebb400-776c-43a5-aa46-578d97933aa1 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0673201 May 20 04:22:34.679969 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3350a7ce-7866-402b-bd03-10f2912846b6 req-46991c82-276d-4d19-885a-8309cdeb9da8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0586791 May 20 04:22:35.141383 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6a4d23b7-b061-4275-9cba-5323521cb448 req-b1a70b50-6622-4d7a-a445-9a1c786b2d69 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:35.142939 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6a4d23b7-b061-4275-9cba-5323521cb448 req-b1a70b50-6622-4d7a-a445-9a1c786b2d69 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.1467612 May 20 04:22:35.151039 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-058979bd-d7ef-4396-a4ea-e23313afa56b req-d690df11-480d-45e1-93dc-9f6a0b25e158 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:35.152395 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-058979bd-d7ef-4396-a4ea-e23313afa56b req-d690df11-480d-45e1-93dc-9f6a0b25e158 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 945 time: 0.1543925 May 20 04:22:35.221939 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6a4d23b7-b061-4275-9cba-5323521cb448 req-ff94a3d5-0b69-4d09-b459-414ab3a9a2dd tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0686202 May 20 04:22:35.226831 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-058979bd-d7ef-4396-a4ea-e23313afa56b req-526a7975-dc6d-4a0e-85f4-b69d7459f4dd tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0610995 May 20 04:22:35.532052 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:35.880097 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4d4aa70e-26b7-49e5-9c7b-8124f0c0fc4d req-122c8b63-bad5-41f9-aa98-94011a6303a0 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:35.881536 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4d4aa70e-26b7-49e5-9c7b-8124f0c0fc4d req-122c8b63-bad5-41f9-aa98-94011a6303a0 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 944 time: 0.0849826 May 20 04:22:35.965386 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4d4aa70e-26b7-49e5-9c7b-8124f0c0fc4d req-93946337-0c8b-40db-9d72-677f58a81477 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0776851 May 20 04:22:36.013188 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eedc479a-8582-423b-8080-681e8bc9cad4 req-0e9b4552-11cd-42b4-bc63-cce208f04caa tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:36.017866 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-eedc479a-8582-423b-8080-681e8bc9cad4 req-0e9b4552-11cd-42b4-bc63-cce208f04caa tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 944 time: 0.0907352 May 20 04:22:36.062804 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-eedc479a-8582-423b-8080-681e8bc9cad4 req-47e5c488-0306-47b9-969d-b8cc2a8242a6 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0410888 May 20 04:22:36.472330 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-184bc455-dc4a-4959-82a2-5074770a6f53 req-c54ac43e-a2d4-4492-9c3f-f03d1b2320d1 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:36.473788 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-184bc455-dc4a-4959-82a2-5074770a6f53 req-c54ac43e-a2d4-4492-9c3f-f03d1b2320d1 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 945 time: 0.1386964 May 20 04:22:36.486195 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7d7caa90-9ee2-481a-a31f-a1a4436ed1f2 req-6d189a5b-c059-4e30-a8b3-24b810748ed7 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:36.487594 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7d7caa90-9ee2-481a-a31f-a1a4436ed1f2 req-6d189a5b-c059-4e30-a8b3-24b810748ed7 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.1213822 May 20 04:22:36.555418 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-184bc455-dc4a-4959-82a2-5074770a6f53 req-f18eca84-ac14-4e3f-a151-edd9a6f4d3b1 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0668263 May 20 04:22:36.562457 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7d7caa90-9ee2-481a-a31f-a1a4436ed1f2 req-523f0260-384e-4eed-889c-ee8652e1dc8b tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0606127 May 20 04:22:37.165494 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8b2931a5-a866-4839-b29b-fcde72288f72 req-f41b3a5e-927d-475a-acc2-68ca16cd220a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:37.174519 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8b2931a5-a866-4839-b29b-fcde72288f72 req-f41b3a5e-927d-475a-acc2-68ca16cd220a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 944 time: 0.0897460 May 20 04:22:37.261597 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8b2931a5-a866-4839-b29b-fcde72288f72 req-306571cd-57c7-4525-aaf2-d1e1f1860c9f tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0888550 May 20 04:22:37.290168 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bd3f2ee4-b91f-4ea7-bcfd-233587f9ed24 req-3d36cdc9-e0af-4d66-841f-4b943ca35588 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:37.291625 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bd3f2ee4-b91f-4ea7-bcfd-233587f9ed24 req-3d36cdc9-e0af-4d66-841f-4b943ca35588 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 944 time: 0.1035545 May 20 04:22:37.340006 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bd3f2ee4-b91f-4ea7-bcfd-233587f9ed24 req-a4e72b57-c08a-4108-83aa-299569ced73f tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0426648 May 20 04:22:37.802926 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ad6967f7-ba17-45c8-8ed1-bdf8d805241e req-56d0f987-9434-4b71-85de-af72dc1830aa tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:37.804505 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ad6967f7-ba17-45c8-8ed1-bdf8d805241e req-56d0f987-9434-4b71-85de-af72dc1830aa tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 945 time: 0.1435106 May 20 04:22:37.814300 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d1c59d81-5a75-49a9-883d-351719ff0459 req-ee003c92-78ed-4a9d-82e1-7aca0d162bd1 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:37.815667 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d1c59d81-5a75-49a9-883d-351719ff0459 req-ee003c92-78ed-4a9d-82e1-7aca0d162bd1 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.1213062 May 20 04:22:37.893107 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ad6967f7-ba17-45c8-8ed1-bdf8d805241e req-dfec2ff9-216f-4499-becf-d3b34b48a8b0 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0764987 May 20 04:22:37.897467 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d1c59d81-5a75-49a9-883d-351719ff0459 req-caee637a-3120-411f-8383-b0d3bd4993f7 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0683584 May 20 04:22:38.453587 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c00d97ae-a17c-4fff-8690-8b45c13b209d req-8954019c-048f-4aaa-871d-9f1e66aa544a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:38.454994 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c00d97ae-a17c-4fff-8690-8b45c13b209d req-8954019c-048f-4aaa-871d-9f1e66aa544a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 944 time: 0.0846655 May 20 04:22:38.603937 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c00d97ae-a17c-4fff-8690-8b45c13b209d req-07bb85b3-83e0-4c1f-8fa1-c48c21026bd1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0534489 May 20 04:22:38.628095 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3fced03b-72db-466e-bbcb-43e4d0f66fe3 req-637f7518-88db-40b4-a8ff-ffc2c7ea81f9 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:38.629900 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3fced03b-72db-466e-bbcb-43e4d0f66fe3 req-637f7518-88db-40b4-a8ff-ffc2c7ea81f9 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 944 time: 0.1740382 May 20 04:22:38.683553 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3fced03b-72db-466e-bbcb-43e4d0f66fe3 req-f1895bdc-2e38-4e3a-85c8-8b271c19e01c tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0465677 May 20 04:22:39.018381 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['849868be-af51-4967-9a6c-93d23cecb450'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:22:39.144470 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2c810a5a-1105-43fa-8b87-d6a7b0e29a8b req-9e1c6278-bf8d-4834-936a-9e335430258d tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:39.145914 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2c810a5a-1105-43fa-8b87-d6a7b0e29a8b req-9e1c6278-bf8d-4834-936a-9e335430258d tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 945 time: 0.1366377 May 20 04:22:39.158116 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2e67df9f-9b13-495b-91d0-feaae25e6853 req-327a7389-bdf5-47fd-8359-98a6d3b877fc tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:39.159607 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2e67df9f-9b13-495b-91d0-feaae25e6853 req-327a7389-bdf5-47fd-8359-98a6d3b877fc tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.1130624 May 20 04:22:39.229577 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2c810a5a-1105-43fa-8b87-d6a7b0e29a8b req-f2778260-d71a-47e5-b15a-f58874bc763e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0689189 May 20 04:22:39.233666 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2e67df9f-9b13-495b-91d0-feaae25e6853 req-424c6bfa-5e56-4de3-ae31-332064513563 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0605550 May 20 04:22:39.717842 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['efb81444-f373-4e9c-a0aa-e3270bd4b980'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:22:39.802230 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:22:39.830087 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-43c36fa1-2e8b-4ebd-accc-a019a374c5d6 req-e9f988c9-6d9e-4623-8adb-92763b90cb3c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:39.831659 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-43c36fa1-2e8b-4ebd-accc-a019a374c5d6 req-e9f988c9-6d9e-4623-8adb-92763b90cb3c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 944 time: 0.0998719 May 20 04:22:39.926311 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-43c36fa1-2e8b-4ebd-accc-a019a374c5d6 req-d33fae40-5bac-4f67-a74a-64e4215f0035 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0584252 May 20 04:22:39.953456 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5041299c-a386-4034-9336-97088975b172 req-1e3df06f-972e-4511-a064-915dea1c1b5a tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:39.955057 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5041299c-a386-4034-9336-97088975b172 req-1e3df06f-972e-4511-a064-915dea1c1b5a tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 944 time: 0.1224372 May 20 04:22:40.005779 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5041299c-a386-4034-9336-97088975b172 req-9b578fdb-0283-4816-8198-fef811c32d30 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0445881 May 20 04:22:40.500193 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b4fbf229-19fd-45cc-8728-6bd05dd9eaee req-f9cefc4e-b74c-4730-ab59-817458cccad0 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:40.501640 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b4fbf229-19fd-45cc-8728-6bd05dd9eaee req-f9cefc4e-b74c-4730-ab59-817458cccad0 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 945 time: 0.1565208 May 20 04:22:40.507785 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['535d5637-ae66-43cb-bd16-3543bd3b7a8b'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:22:40.510424 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fa63ffe7-cb43-4f86-b190-f7059f49855a req-fcedd2cd-6c71-4ccc-a2c9-e807703fa8bd tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:40.511839 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fa63ffe7-cb43-4f86-b190-f7059f49855a req-fcedd2cd-6c71-4ccc-a2c9-e807703fa8bd tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.1306076 May 20 04:22:40.581171 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b4fbf229-19fd-45cc-8728-6bd05dd9eaee req-c0bc2163-5166-407c-85b4-5417f55c9f04 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0683382 May 20 04:22:40.588544 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fa63ffe7-cb43-4f86-b190-f7059f49855a req-0c5d1edf-bc3f-464f-bbe2-382fff91544e tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0626485 May 20 04:22:41.128001 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e5b3a12a-5fef-49f2-a409-ce9a6c3e9e85 req-7dda9237-592f-437c-9300-669130da765f tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:41.131793 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e5b3a12a-5fef-49f2-a409-ce9a6c3e9e85 req-7dda9237-592f-437c-9300-669130da765f tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 944 time: 0.0917237 May 20 04:22:41.226738 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e5b3a12a-5fef-49f2-a409-ce9a6c3e9e85 req-1a96c003-c519-4804-9838-9cca72b39e8a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0595777 May 20 04:22:41.246065 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4bc657b4-bc94-491f-8073-972327e1f29c req-cfa30171-9690-4ed8-8488-bab864b6b616 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:41.251247 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4bc657b4-bc94-491f-8073-972327e1f29c req-cfa30171-9690-4ed8-8488-bab864b6b616 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 944 time: 0.1148491 May 20 04:22:41.300016 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4bc657b4-bc94-491f-8073-972327e1f29c req-23517693-860d-4a39-87d3-6136d7c7576c tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0419457 May 20 04:22:41.833013 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ea65c8aa-fdeb-4079-8bea-300c2c2a3b9e req-712b2bf6-519a-439b-b2af-e69999a944d8 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:41.834433 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ea65c8aa-fdeb-4079-8bea-300c2c2a3b9e req-712b2bf6-519a-439b-b2af-e69999a944d8 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 945 time: 0.1418581 May 20 04:22:41.850931 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-632907f3-3720-40fe-afbc-b0354dfd15a4 req-f5749765-b23f-496a-86fa-56f991778139 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:41.852444 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-632907f3-3720-40fe-afbc-b0354dfd15a4 req-f5749765-b23f-496a-86fa-56f991778139 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.1230624 May 20 04:22:41.918117 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ea65c8aa-fdeb-4079-8bea-300c2c2a3b9e req-671a908d-17ea-4987-826c-b8e84036cd19 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0646949 May 20 04:22:41.928564 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-632907f3-3720-40fe-afbc-b0354dfd15a4 req-1ffa96b7-23e0-4f7a-a8b2-5137680135ee tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0601163 May 20 04:22:42.473810 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d617226b-ddd7-4c8f-ae82-77a01845e89a req-e6f7532d-c245-44a9-8771-7be1df6cda53 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:42.475244 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d617226b-ddd7-4c8f-ae82-77a01845e89a req-e6f7532d-c245-44a9-8771-7be1df6cda53 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 944 time: 0.1269555 May 20 04:22:42.528940 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d617226b-ddd7-4c8f-ae82-77a01845e89a req-2da67ad0-6cd1-4b40-b834-b4bc33e6191f tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0466368 May 20 04:22:42.538953 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2ccc11fc-2c74-4d01-8de9-07dddd995bcb req-797172f9-6ba7-4b9f-b5cf-d6ad6a1a639c tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:42.540388 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2ccc11fc-2c74-4d01-8de9-07dddd995bcb req-797172f9-6ba7-4b9f-b5cf-d6ad6a1a639c tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 944 time: 0.1290393 May 20 04:22:42.587311 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2ccc11fc-2c74-4d01-8de9-07dddd995bcb req-99e860c6-64fa-4b45-96a0-2a13e102721c tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0412779 May 20 04:22:42.598352 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c7a9a466-21d4-439e-86da-a8d78bdc68b0 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:43.170736 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-32a7552d-3907-4fb2-b9fb-8b341ef82aaf req-e52a95d7-203b-46ec-82d3-004843d594d7 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:43.172194 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-32a7552d-3907-4fb2-b9fb-8b341ef82aaf req-e52a95d7-203b-46ec-82d3-004843d594d7 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 945 time: 0.1379755 May 20 04:22:43.182162 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4bc07c81-780a-4740-bac4-a45d151d3ee1 req-80d16fbd-622a-49f9-966c-7e01022498e4 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:43.183506 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4bc07c81-780a-4740-bac4-a45d151d3ee1 req-80d16fbd-622a-49f9-966c-7e01022498e4 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.1132579 May 20 04:22:43.253318 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-32a7552d-3907-4fb2-b9fb-8b341ef82aaf req-06e4ab43-1ea0-41fc-8d1d-8afb134e2d4d tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0688071 May 20 04:22:43.255736 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4bc07c81-780a-4740-bac4-a45d151d3ee1 req-6a1fda24-6946-4ba1-8689-4be1eeac8af1 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0589421 May 20 04:22:43.771441 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1fc53fc5-78f7-4d35-9964-20b597603417 req-d372c35b-93f8-4503-baf1-a173b33cd9f6 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:43.772949 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1fc53fc5-78f7-4d35-9964-20b597603417 req-d372c35b-93f8-4503-baf1-a173b33cd9f6 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 944 time: 0.1319492 May 20 04:22:43.880748 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-25037ed8-e778-470c-b5ad-09acac74231d req-71a5c9e7-b524-491a-8b3e-c4c63418752a tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:43.882640 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-25037ed8-e778-470c-b5ad-09acac74231d req-71a5c9e7-b524-491a-8b3e-c4c63418752a tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 944 time: 0.1844244 May 20 04:22:43.889570 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1fc53fc5-78f7-4d35-9964-20b597603417 req-bae76f6a-50b1-4881-9ed4-87bb26eb688c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1098955 May 20 04:22:43.929533 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-25037ed8-e778-470c-b5ad-09acac74231d req-6e376096-d933-4168-bf27-a6c43ab198a2 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0390429 May 20 04:22:44.501527 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0f7c1f1-9ee4-4222-9981-d804778490a9 req-1bc274b1-f89a-4cce-a66d-7c4ffb4ea22c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:44.502961 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d0f7c1f1-9ee4-4222-9981-d804778490a9 req-1bc274b1-f89a-4cce-a66d-7c4ffb4ea22c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 945 time: 0.1415620 May 20 04:22:44.510921 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a480ae43-d8a7-49f7-b44c-daab01d09132 req-6516ae8b-0a76-4cbc-af24-977df9f8c549 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:44.512377 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a480ae43-d8a7-49f7-b44c-daab01d09132 req-6516ae8b-0a76-4cbc-af24-977df9f8c549 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.1197596 May 20 04:22:44.583489 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d0f7c1f1-9ee4-4222-9981-d804778490a9 req-a60ddc36-c739-4681-a464-3006790b1ec3 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0701399 May 20 04:22:44.588110 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a480ae43-d8a7-49f7-b44c-daab01d09132 req-b2dbfb71-23d4-4ee6-933a-0f5673ab0b49 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0618563 May 20 04:22:44.963734 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:45.076483 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e138b23e-8cea-4802-a40e-c22d1e63b0ae None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:45.158289 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4fbb90ea-f527-4330-83e2-10544d3df9a8 req-170dbd44-53db-41b3-a481-8bc5d1e9cf87 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:45.159903 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4fbb90ea-f527-4330-83e2-10544d3df9a8 req-170dbd44-53db-41b3-a481-8bc5d1e9cf87 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 944 time: 0.1109037 May 20 04:22:45.168385 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-34f0c4c1-d385-4803-b4ac-a3dd6a237676 req-1f514f14-7890-4c3f-8349-8e34c63799ed tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:45.169769 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-34f0c4c1-d385-4803-b4ac-a3dd6a237676 req-1f514f14-7890-4c3f-8349-8e34c63799ed tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 944 time: 0.1537492 May 20 04:22:45.243117 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4fbb90ea-f527-4330-83e2-10544d3df9a8 req-e12497c6-0047-43b5-b6c7-b3b972ecb9df tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0724254 May 20 04:22:45.250614 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-34f0c4c1-d385-4803-b4ac-a3dd6a237676 req-79cdbdef-ac5b-4802-8610-8703b989bcd3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0636570 May 20 04:22:45.845141 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-97e42c01-5a54-4e5c-949b-4bf2defe3a75 req-90c4f0b7-1e88-49b6-ba8b-fb18f2940a9c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:45.846704 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-97e42c01-5a54-4e5c-949b-4bf2defe3a75 req-90c4f0b7-1e88-49b6-ba8b-fb18f2940a9c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 945 time: 0.1143818 May 20 04:22:45.854376 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4027a9db-f21d-4640-88aa-3a22dcf38bcc req-47e5ab81-fab2-46f8-b6b7-e39bb3c44cf6 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:45.855849 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4027a9db-f21d-4640-88aa-3a22dcf38bcc req-47e5ab81-fab2-46f8-b6b7-e39bb3c44cf6 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.1567733 May 20 04:22:45.925570 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-97e42c01-5a54-4e5c-949b-4bf2defe3a75 req-d565592d-1d80-44fc-8406-95a2bac7b555 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0687282 May 20 04:22:45.932615 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4027a9db-f21d-4640-88aa-3a22dcf38bcc req-a5a709d8-c5d3-4db9-8a1b-1d3ad37297c5 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0628788 May 20 04:22:46.163653 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:46.492516 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-19351a6e-f4bc-43d3-8b0e-58ef3d8941d8 req-ccdc89e0-06c8-46ef-b0e8-ca165797fa8b tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:46.493891 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-19351a6e-f4bc-43d3-8b0e-58ef3d8941d8 req-ccdc89e0-06c8-46ef-b0e8-ca165797fa8b tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 944 time: 0.1430089 May 20 04:22:46.504043 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9575fcc3-7631-47f9-81f4-2650f8d3b819 req-71df2b7f-5f23-457e-bacb-cec83f50a8a6 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:46.505669 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9575fcc3-7631-47f9-81f4-2650f8d3b819 req-71df2b7f-5f23-457e-bacb-cec83f50a8a6 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 944 time: 0.1229589 May 20 04:22:46.577716 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-19351a6e-f4bc-43d3-8b0e-58ef3d8941d8 req-3c34a219-6606-443b-ad5a-a55841cadf14 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0709946 May 20 04:22:46.584403 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9575fcc3-7631-47f9-81f4-2650f8d3b819 req-0218afd3-a758-4f10-9596-c003fc26498b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0648880 May 20 04:22:47.178824 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c14b9f41-8ddd-4b6c-8222-cbf0677cef1e req-6b3e26c9-bdf6-460c-9042-7c45550de1c7 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:47.180336 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c14b9f41-8ddd-4b6c-8222-cbf0677cef1e req-6b3e26c9-bdf6-460c-9042-7c45550de1c7 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 945 time: 0.1399357 May 20 04:22:47.192531 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f8fbec83-67ed-49d2-af9a-69199e3634bd req-afed7016-0791-478f-b1b7-9b3b441d5a7c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:47.193847 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f8fbec83-67ed-49d2-af9a-69199e3634bd req-afed7016-0791-478f-b1b7-9b3b441d5a7c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.1203012 May 20 04:22:47.267092 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c14b9f41-8ddd-4b6c-8222-cbf0677cef1e req-d1aadf6f-2864-4d55-a156-52c36cb62165 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0718427 May 20 04:22:47.274040 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f8fbec83-67ed-49d2-af9a-69199e3634bd req-c888c22f-36eb-4f89-afd7-9bd965dc6a0f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0608518 May 20 04:22:47.790554 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:47.823026 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6269f016-e920-4512-92bd-68fd79065ac1 req-3303beb2-95f7-4f88-8cd2-ef0379614435 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:47.824408 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6269f016-e920-4512-92bd-68fd79065ac1 req-3303beb2-95f7-4f88-8cd2-ef0379614435 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 944 time: 0.1407282 May 20 04:22:47.835934 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4088c0da-b4c4-427b-bbb0-6f4b988818db req-622be397-cf1a-4cde-8189-9ab620cf99b8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:47.837278 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4088c0da-b4c4-427b-bbb0-6f4b988818db req-622be397-cf1a-4cde-8189-9ab620cf99b8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 944 time: 0.1232471 May 20 04:22:47.904410 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6269f016-e920-4512-92bd-68fd79065ac1 req-5d226f98-3c12-4806-bdbc-cda1646a1be8 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0662246 May 20 04:22:47.908739 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4088c0da-b4c4-427b-bbb0-6f4b988818db req-8f3d668a-6b6a-484d-bfa4-256860f95eeb tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0577445 May 20 04:22:48.317536 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device a8b0c56a-171d-40e9-beb7-c7b63fa031b9 details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:22:48.361487 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 849868be-af51-4967-9a6c-93d23cecb450 up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:22:48.521330 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port a8b0c56a-171d-40e9-beb7-c7b63fa031b9, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=a8b0c56a-171d-40e9-beb7-c7b63fa031b9,segment=NetworkSegment(0032d9a3-07bf-489f-bb9d-03ab3ff14f98),segment_id=0032d9a3-07bf-489f-bb9d-03ab3ff14f98)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:22:48.528901 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port 849868be-af51-4967-9a6c-93d23cecb450 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:22:48.529025 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c7fb2de5-d495-4cf2-8926-f1a735e97251 req-5dcf5c23-bbe6-4b76-8c17-f0275ee71ba7 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:48.530441 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c7fb2de5-d495-4cf2-8926-f1a735e97251 req-5dcf5c23-bbe6-4b76-8c17-f0275ee71ba7 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 944 time: 0.1128318 May 20 04:22:48.539476 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': 'a8b0c56a-171d-40e9-beb7-c7b63fa031b9', 'network_id': '36b37213-8439-447d-9eba-a78baf7c698e', 'port_id': 'a8b0c56a-171d-40e9-beb7-c7b63fa031b9', 'mac_address': 'fa:16:3e:7b:1d:ca', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 813, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'b25ae9c8-e11d-45ee-bde2-332cc1538602', 'ip_address': '172.20.0.13'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:22:48.541629 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port 849868be-af51-4967-9a6c-93d23cecb450 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:22:48.542196 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:48.549367 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ebe1e567-0a48-4a9d-ac11-7fa5f2ffb5eb req-cbb32e12-a09b-46df-a2f4-78481322b662 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:48.551095 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ebe1e567-0a48-4a9d-ac11-7fa5f2ffb5eb req-cbb32e12-a09b-46df-a2f4-78481322b662 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 945 time: 0.1718929 May 20 04:22:48.632953 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c7fb2de5-d495-4cf2-8926-f1a735e97251 req-a0a6d363-a6d6-4609-9407-a539475a1d49 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0924900 May 20 04:22:48.647792 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ebe1e567-0a48-4a9d-ac11-7fa5f2ffb5eb req-e003d5cb-1c7d-4a80-8b88-1eb2a5c2fd8f tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0785248 May 20 04:22:48.656745 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:48.678029 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id a8b0c56a-171d-40e9-beb7-c7b63fa031b9 {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:22:48.734423 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port a8b0c56a-171d-40e9-beb7-c7b63fa031b9, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=a8b0c56a-171d-40e9-beb7-c7b63fa031b9,segment=NetworkSegment(0032d9a3-07bf-489f-bb9d-03ab3ff14f98),segment_id=0032d9a3-07bf-489f-bb9d-03ab3ff14f98)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:22:48.740105 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:48.743766 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:48.789231 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:48.831383 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'e94a8b2b-8a29-45d4-9edb-63137be69ce4', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '849868be-af51-4967-9a6c-93d23cecb450'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:22:48.892233 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=a8b0c56a-171d-40e9-beb7-c7b63fa031b9,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:48.901883 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 849868be-af51-4967-9a6c-93d23cecb450, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=849868be-af51-4967-9a6c-93d23cecb450,segment=NetworkSegment(6fe99325-a72c-48bf-abc7-9bbcc4b753ff),segment_id=6fe99325-a72c-48bf-abc7-9bbcc4b753ff)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:22:48.908146 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:48.908648 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:48.909370 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:48.909804 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:48.910286 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:48.910763 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:48.911277 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:48.911853 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:48.912330 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:48.912844 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:48.913269 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:48.913576 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:48.917402 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:48.919181 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.175s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:48.919672 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:49.000207 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=849868be-af51-4967-9a6c-93d23cecb450,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:49.003908 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:49.165156 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b8d54773-6071-4e48-8c80-b20de4790fda req-27875297-de5b-4766-9d00-7e4ed69dac57 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:49.166498 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b8d54773-6071-4e48-8c80-b20de4790fda req-27875297-de5b-4766-9d00-7e4ed69dac57 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 944 time: 0.1387937 May 20 04:22:49.174360 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-61181be2-4d0e-4ef2-b53f-edde3063748d req-db12abbb-3b32-496f-a4f6-e60ff9c9ec00 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:49.175197 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-61181be2-4d0e-4ef2-b53f-edde3063748d req-db12abbb-3b32-496f-a4f6-e60ff9c9ec00 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 944 time: 0.1459324 May 20 04:22:49.312120 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b8d54773-6071-4e48-8c80-b20de4790fda req-be540120-08ce-4570-9bdf-93ba91d42c52 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1359351 May 20 04:22:49.318912 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-61181be2-4d0e-4ef2-b53f-edde3063748d req-1274f246-24d9-4743-aacf-5cccd5e07459 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1257370 May 20 04:22:49.423462 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'e94a8b2b-8a29-45d4-9edb-63137be69ce4', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '849868be-af51-4967-9a6c-93d23cecb450', 'code': 200} May 20 04:22:49.662781 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-441c6923-576e-491a-87f7-3ec7a7ea607c req-5f29a00f-501b-4860-86af-10abca0e9b8f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:49.664306 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-441c6923-576e-491a-87f7-3ec7a7ea607c req-5f29a00f-501b-4860-86af-10abca0e9b8f service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 1099 time: 0.0723770 May 20 04:22:49.687608 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-441c6923-576e-491a-87f7-3ec7a7ea607c req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:49.703235 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-441c6923-576e-491a-87f7-3ec7a7ea607c req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:22:49.707299 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-441c6923-576e-491a-87f7-3ec7a7ea607c req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Usage tracker for resource:security_group_rule and tenant:11f8ac5c97ce4c03adee5db9032a8fb4 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:22:49.710317 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-441c6923-576e-491a-87f7-3ec7a7ea607c req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:22:49.716009 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-441c6923-576e-491a-87f7-3ec7a7ea607c req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:22:49.806841 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-441c6923-576e-491a-87f7-3ec7a7ea607c req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:49.812605 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-441c6923-576e-491a-87f7-3ec7a7ea607c req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:49.837859 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:49.875963 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-441c6923-576e-491a-87f7-3ec7a7ea607c req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2047782 May 20 04:22:49.883997 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Pushing event updated for resources: {'SecurityGroup': ['ID=8bf73c55-faaa-4777-9be5-da3d19ec427f,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:49.893604 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:49.894191 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:49.894700 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:49.895197 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:49.895728 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:49.896207 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:49.896695 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:49.897230 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:49.897709 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:49.898215 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:49.899078 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:49.899654 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:49.913818 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-61286f97-15ae-493e-a2d2-cbea513a1f12 req-6f796079-5b80-4a8e-bf9c-415d16549173 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:49.915796 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:49.921131 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-61286f97-15ae-493e-a2d2-cbea513a1f12 req-6f796079-5b80-4a8e-bf9c-415d16549173 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 945 time: 0.1756525 May 20 04:22:49.979640 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7026d382-7411-48c2-a9de-145c82919e9a req-2a332753-09e3-42d5-a6db-06086c843d6e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:49.981071 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7026d382-7411-48c2-a9de-145c82919e9a req-2a332753-09e3-42d5-a6db-06086c843d6e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 946 time: 0.1634879 May 20 04:22:49.983834 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-61286f97-15ae-493e-a2d2-cbea513a1f12 req-ccbd74db-1fd4-47c4-8a6d-2e996ccd70dd tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0618813 May 20 04:22:50.006319 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['a8b0c56a-171d-40e9-beb7-c7b63fa031b9'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:22:50.041656 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7026d382-7411-48c2-a9de-145c82919e9a req-e12fd8ab-d21a-4dbc-b94c-6612c030463b tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0483072 May 20 04:22:50.296060 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 706277c6-cd8c-4146-ab91-ed9504446690 details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:22:50.299498 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device efb81444-f373-4e9c-a0aa-e3270bd4b980 up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:22:50.462230 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port efb81444-f373-4e9c-a0aa-e3270bd4b980 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:22:50.501700 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port efb81444-f373-4e9c-a0aa-e3270bd4b980 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:22:50.502072 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:50.574552 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 706277c6-cd8c-4146-ab91-ed9504446690, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=706277c6-cd8c-4146-ab91-ed9504446690,segment=NetworkSegment(0032d9a3-07bf-489f-bb9d-03ab3ff14f98),segment_id=0032d9a3-07bf-489f-bb9d-03ab3ff14f98)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:22:50.575948 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ffd272f7-5e5b-4d1b-80e3-1503f0ae5468 req-be6afa7c-9026-446a-9d26-c70f31c150e8 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:50.577371 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ffd272f7-5e5b-4d1b-80e3-1503f0ae5468 req-be6afa7c-9026-446a-9d26-c70f31c150e8 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 944 time: 0.1394680 May 20 04:22:50.590754 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': '706277c6-cd8c-4146-ab91-ed9504446690', 'network_id': '36b37213-8439-447d-9eba-a78baf7c698e', 'port_id': '706277c6-cd8c-4146-ab91-ed9504446690', 'mac_address': 'fa:16:3e:1e:db:f1', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 813, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'b25ae9c8-e11d-45ee-bde2-332cc1538602', 'ip_address': '172.20.0.8'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:22:50.607796 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8a0bdd13-d452-4329-8e18-e97c811e11ec req-8f3b7cbb-10d7-4e87-9d3e-baea8b4f95ec tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:50.609199 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8a0bdd13-d452-4329-8e18-e97c811e11ec req-8f3b7cbb-10d7-4e87-9d3e-baea8b4f95ec tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 944 time: 0.1307893 May 20 04:22:50.663595 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ffd272f7-5e5b-4d1b-80e3-1503f0ae5468 req-bb005769-a4ee-4a70-ba15-bb4c997c450b tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0790172 May 20 04:22:50.674734 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8a0bdd13-d452-4329-8e18-e97c811e11ec req-84cff40b-6dff-4f0a-9455-257f8cee02bb tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0590708 May 20 04:22:50.698238 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:50.722704 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 706277c6-cd8c-4146-ab91-ed9504446690 {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:22:50.726649 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:50.758114 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 706277c6-cd8c-4146-ab91-ed9504446690, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=706277c6-cd8c-4146-ab91-ed9504446690,segment=NetworkSegment(0032d9a3-07bf-489f-bb9d-03ab3ff14f98),segment_id=0032d9a3-07bf-489f-bb9d-03ab3ff14f98)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:22:50.765315 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:50.768363 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:50.800388 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port efb81444-f373-4e9c-a0aa-e3270bd4b980, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=efb81444-f373-4e9c-a0aa-e3270bd4b980,segment=NetworkSegment(3c4a2238-8a32-4d92-a1b1-8af91c1c27a7),segment_id=3c4a2238-8a32-4d92-a1b1-8af91c1c27a7)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:22:50.806651 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:50.807572 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 535d5637-ae66-43cb-bd16-3543bd3b7a8b up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:22:50.888596 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=706277c6-cd8c-4146-ab91-ed9504446690,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:50.891739 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.123s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:50.892130 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.053s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:50.958322 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port 535d5637-ae66-43cb-bd16-3543bd3b7a8b completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:22:50.966705 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port 535d5637-ae66-43cb-bd16-3543bd3b7a8b triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:22:50.967420 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:50.991206 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=efb81444-f373-4e9c-a0aa-e3270bd4b980,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:51.001892 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.110s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:51.098576 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:51.163906 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 535d5637-ae66-43cb-bd16-3543bd3b7a8b, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=535d5637-ae66-43cb-bd16-3543bd3b7a8b,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:22:51.168988 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:51.171426 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-8eed0b07-ca9a-4558-ad1c-f21ff862419c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:51.272188 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=535d5637-ae66-43cb-bd16-3543bd3b7a8b,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:51.275626 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-93133d14-086c-4ef9-9012-3e4e960537db req-4d3e887f-c0ce-476f-a906-c2ca9d92feeb tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:51.276950 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.106s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:51.282516 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-93133d14-086c-4ef9-9012-3e4e960537db req-4d3e887f-c0ce-476f-a906-c2ca9d92feeb tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 945 time: 0.1752424 May 20 04:22:51.332537 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2261bbea-a340-44e8-86f0-d505adb413eb req-ab24dd76-ff34-4f0b-8ffb-9dfcf2ee902b tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:51.333977 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2261bbea-a340-44e8-86f0-d505adb413eb req-ab24dd76-ff34-4f0b-8ffb-9dfcf2ee902b tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 946 time: 0.1526883 May 20 04:22:51.339181 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-93133d14-086c-4ef9-9012-3e4e960537db req-a86e4fb0-6af2-42fb-a426-32b50418000d tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0558760 May 20 04:22:51.398739 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2261bbea-a340-44e8-86f0-d505adb413eb req-97333e55-b637-408f-8d05-d4e6b490ffa1 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0552781 May 20 04:22:51.422355 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['706277c6-cd8c-4146-ab91-ed9504446690'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:22:51.483305 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'bea4be17-559d-407e-9b61-ff2cb94317dc', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'efb81444-f373-4e9c-a0aa-e3270bd4b980'}, {'server_uuid': 'c2ca950f-78ff-4e44-9231-e8e19f4aee5d', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '535d5637-ae66-43cb-bd16-3543bd3b7a8b'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:22:51.535498 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'bea4be17-559d-407e-9b61-ff2cb94317dc', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'efb81444-f373-4e9c-a0aa-e3270bd4b980', 'code': 200} May 20 04:22:51.535772 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'c2ca950f-78ff-4e44-9231-e8e19f4aee5d', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '535d5637-ae66-43cb-bd16-3543bd3b7a8b', 'code': 200} May 20 04:22:51.620190 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 34622) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:51.770137 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-602010bf-d62e-4e24-b090-392bcbe40f2e admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:51.788834 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-602010bf-d62e-4e24-b090-392bcbe40f2e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:22:51.793406 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-602010bf-d62e-4e24-b090-392bcbe40f2e admin admin] Usage tracker for resource:security_group_rule and tenant:c98a96630f7b4f5b893002d587e547c9 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:22:51.796572 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-602010bf-d62e-4e24-b090-392bcbe40f2e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:22:51.802709 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-602010bf-d62e-4e24-b090-392bcbe40f2e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:22:51.982497 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9dc1a341-824d-4c7d-90cb-16baf3d85a92 req-740ed095-0239-4aa6-a508-affddb15a5e6 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:51.984118 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9dc1a341-824d-4c7d-90cb-16baf3d85a92 req-740ed095-0239-4aa6-a508-affddb15a5e6 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.1654902 May 20 04:22:51.994323 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f25f24bc-5e20-4228-8757-b9845f69ae93 req-5e41b44f-60da-4b3a-9043-3c26f099660d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:51.995771 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f25f24bc-5e20-4228-8757-b9845f69ae93 req-5e41b44f-60da-4b3a-9043-3c26f099660d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1291966 May 20 04:22:52.007009 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-602010bf-d62e-4e24-b090-392bcbe40f2e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:52.016246 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-602010bf-d62e-4e24-b090-392bcbe40f2e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:52.026458 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-4259269b-8b1c-444e-ad15-e7f3cf69ff00 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:52.061456 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1bbfe5ea-8234-47bc-ac87-988247c26f02 req-c1899c0c-d643-4312-b688-99597b26b98b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:52.062890 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-1bbfe5ea-8234-47bc-ac87-988247c26f02 req-c1899c0c-d643-4312-b688-99597b26b98b service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 1098 time: 0.2217989 May 20 04:22:52.063035 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9dc1a341-824d-4c7d-90cb-16baf3d85a92 req-1e718109-eea1-4655-b527-65b1f0efb29e tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0661831 May 20 04:22:52.070295 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f25f24bc-5e20-4228-8757-b9845f69ae93 req-b5ee61c4-b2a2-4650-81a5-c8d7a0b58bbd tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0599809 May 20 04:22:52.071185 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d2a9d229-42e4-4706-a017-e9ddec218372 req-20724b13-9720-43cf-8c93-6ba1c73c5854 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:52.072608 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-d2a9d229-42e4-4706-a017-e9ddec218372 req-20724b13-9720-43cf-8c93-6ba1c73c5854 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 1098 time: 0.1963224 May 20 04:22:52.089768 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-602010bf-d62e-4e24-b090-392bcbe40f2e admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=c98a96630f7b4f5b893002d587e547c9&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4597611 May 20 04:22:52.090067 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 34634) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:52.108693 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-602010bf-d62e-4e24-b090-392bcbe40f2e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=cdfc3bb6-6ae8-417a-b170-4a28ad82c75d,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:52.111729 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-602010bf-d62e-4e24-b090-392bcbe40f2e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.085s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:52.148148 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-1bbfe5ea-8234-47bc-ac87-988247c26f02 req-705788bd-eef8-4545-b35d-ffe56f03665f service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0723300 May 20 04:22:52.151306 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-d2a9d229-42e4-4706-a017-e9ddec218372 req-fedf0c3f-c5da-41e9-ba7c-d1a74405a187 service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0558345 May 20 04:22:52.151833 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-227f2247-4500-4943-936d-b55e94501b75 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:52.177196 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-227f2247-4500-4943-936d-b55e94501b75 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:52.232815 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-227f2247-4500-4943-936d-b55e94501b75 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:52.236378 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:52.250783 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-227f2247-4500-4943-936d-b55e94501b75 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=cdfc3bb6-6ae8-417a-b170-4a28ad82c75d,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:52.261836 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-227f2247-4500-4943-936d-b55e94501b75 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:52.262188 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-227f2247-4500-4943-936d-b55e94501b75 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:52.262532 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-227f2247-4500-4943-936d-b55e94501b75 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:52.262853 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-227f2247-4500-4943-936d-b55e94501b75 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:52.263188 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-227f2247-4500-4943-936d-b55e94501b75 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:52.263547 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-227f2247-4500-4943-936d-b55e94501b75 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:52.263894 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-227f2247-4500-4943-936d-b55e94501b75 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:52.264220 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-227f2247-4500-4943-936d-b55e94501b75 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:52.264555 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-227f2247-4500-4943-936d-b55e94501b75 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:52.264866 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-227f2247-4500-4943-936d-b55e94501b75 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:52.265190 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-227f2247-4500-4943-936d-b55e94501b75 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:52.265516 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-227f2247-4500-4943-936d-b55e94501b75 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:22:52.270268 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-227f2247-4500-4943-936d-b55e94501b75 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.034s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:52.272414 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-227f2247-4500-4943-936d-b55e94501b75 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:52.273485 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-227f2247-4500-4943-936d-b55e94501b75 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/cdfc3bb6-6ae8-417a-b170-4a28ad82c75d HTTP/1.1" status: 204 len: 168 time: 0.1889129 May 20 04:22:52.531167 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b22ea1b9-2329-4813-a6e0-6739a4f67a4e req-3b110141-bfd8-4340-96e0-ec8151348d97 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:52.538162 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b22ea1b9-2329-4813-a6e0-6739a4f67a4e req-3b110141-bfd8-4340-96e0-ec8151348d97 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 945 time: 0.0915701 May 20 04:22:52.597522 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 34648) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:52.618760 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4d6f54e9-ec7d-4c97-adcd-184fc0c47ed1 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:52.632845 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-4d6f54e9-ec7d-4c97-adcd-184fc0c47ed1 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:22:52.636542 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-4d6f54e9-ec7d-4c97-adcd-184fc0c47ed1 admin admin] Usage tracker for resource:security_group_rule and tenant:3d950048d9a8490c93d70c65ef77e4a9 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:22:52.639251 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-4d6f54e9-ec7d-4c97-adcd-184fc0c47ed1 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:22:52.641735 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b22ea1b9-2329-4813-a6e0-6739a4f67a4e req-5c1a6500-eeef-4f65-993e-05fc5c07b46e tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0968330 May 20 04:22:52.645804 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-4d6f54e9-ec7d-4c97-adcd-184fc0c47ed1 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:22:52.672611 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-aa2bdb89-c491-4e15-bb50-e97835a10392 req-61655954-15c2-4a0b-b228-50cf90ecf6ba tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:52.674116 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-aa2bdb89-c491-4e15-bb50-e97835a10392 req-61655954-15c2-4a0b-b228-50cf90ecf6ba tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 946 time: 0.1319680 May 20 04:22:52.726964 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-aa2bdb89-c491-4e15-bb50-e97835a10392 req-886478f6-1b03-467d-9b42-a3d06bbcc434 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0455742 May 20 04:22:52.733036 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4d6f54e9-ec7d-4c97-adcd-184fc0c47ed1 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:52.737866 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4d6f54e9-ec7d-4c97-adcd-184fc0c47ed1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:52.744931 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-552e0c96-f2af-4537-9895-416f719f34d5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:52.784737 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-4d6f54e9-ec7d-4c97-adcd-184fc0c47ed1 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=3d950048d9a8490c93d70c65ef77e4a9&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1860030 May 20 04:22:52.785395 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4d6f54e9-ec7d-4c97-adcd-184fc0c47ed1 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=47a2424e-6d3a-49e4-b730-feb34c1a4c11,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:52.787889 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 34654) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:22:52.788410 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-4d6f54e9-ec7d-4c97-adcd-184fc0c47ed1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:52.852928 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2afec35f-c57d-4b64-9471-452a8fe09c8c admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:52.878780 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2afec35f-c57d-4b64-9471-452a8fe09c8c admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:52.934378 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2afec35f-c57d-4b64-9471-452a8fe09c8c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:52.939191 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:52.951536 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2afec35f-c57d-4b64-9471-452a8fe09c8c admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=47a2424e-6d3a-49e4-b730-feb34c1a4c11,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:52.960558 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-2afec35f-c57d-4b64-9471-452a8fe09c8c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:52.960558 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2afec35f-c57d-4b64-9471-452a8fe09c8c admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:52.960558 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-2afec35f-c57d-4b64-9471-452a8fe09c8c admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/47a2424e-6d3a-49e4-b730-feb34c1a4c11 HTTP/1.1" status: 204 len: 168 time: 0.1703241 May 20 04:22:53.324858 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9af41fdc-8b37-427e-9385-1c08af4ebbf5 req-dc77c1f7-c980-465b-9fbb-d538507404bc tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:53.326329 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9af41fdc-8b37-427e-9385-1c08af4ebbf5 req-dc77c1f7-c980-465b-9fbb-d538507404bc tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.1428421 May 20 04:22:53.334561 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ba10ad64-7579-4568-9ea8-9f92a8091ada req-71e7e08d-dc46-4908-b9d6-bf7403865388 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:53.339091 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ba10ad64-7579-4568-9ea8-9f92a8091ada req-71e7e08d-dc46-4908-b9d6-bf7403865388 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1154370 May 20 04:22:53.405612 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9af41fdc-8b37-427e-9385-1c08af4ebbf5 req-77a74c3f-ab2e-431c-b6a9-d0ee7e01bd23 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0686839 May 20 04:22:53.410078 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ba10ad64-7579-4568-9ea8-9f92a8091ada req-2c0a8b78-6f52-4892-9bb6-97f6e3609bea tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0582325 May 20 04:22:53.839879 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8a6418e4-5f62-43c7-8747-6d10c29c2eac req-78ecd61a-506c-4c90-b5fc-bdc17ec1987b tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:53.841703 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8a6418e4-5f62-43c7-8747-6d10c29c2eac req-78ecd61a-506c-4c90-b5fc-bdc17ec1987b tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 945 time: 0.0843248 May 20 04:22:53.892868 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8a6418e4-5f62-43c7-8747-6d10c29c2eac req-a92a247c-ac35-4618-b1ca-a8192fc7c4b4 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0443559 May 20 04:22:54.055720 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9f1ae130-6eba-41fb-85e4-48eb820d5368 req-cd963aab-dbdd-46db-b322-33f4130faec0 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:54.057179 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9f1ae130-6eba-41fb-85e4-48eb820d5368 req-cd963aab-dbdd-46db-b322-33f4130faec0 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 946 time: 0.0851181 May 20 04:22:54.166305 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9f1ae130-6eba-41fb-85e4-48eb820d5368 req-1b889018-c99b-4adb-9a98-87d764c4f1fb tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1028600 May 20 04:22:54.672040 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-39edc34c-1a6c-4d2f-8e16-0e0f692e1e0f req-b2f9b444-a13b-4b9f-9be5-3860f8299ca2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:54.673430 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-39edc34c-1a6c-4d2f-8e16-0e0f692e1e0f req-b2f9b444-a13b-4b9f-9be5-3860f8299ca2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1442714 May 20 04:22:54.680725 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-60fc063f-dfb4-43d5-9298-4418ed4cf3ae req-d19cd03d-a35e-4937-812d-7c1548b8b92f tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:54.682084 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-60fc063f-dfb4-43d5-9298-4418ed4cf3ae req-d19cd03d-a35e-4937-812d-7c1548b8b92f tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.1558182 May 20 04:22:54.750730 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-39edc34c-1a6c-4d2f-8e16-0e0f692e1e0f req-bda10afa-1c08-4f96-a7ff-11e8074bc553 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0677223 May 20 04:22:54.753393 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-60fc063f-dfb4-43d5-9298-4418ed4cf3ae req-d9d2786b-0d72-4196-b508-3f5f7053abe2 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0579245 May 20 04:22:55.098924 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-53652dea-2173-4b30-aecd-47e955e8634e req-99389052-3cef-4905-9f55-543faf36ec2c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:55.100357 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-53652dea-2173-4b30-aecd-47e955e8634e req-99389052-3cef-4905-9f55-543faf36ec2c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 945 time: 0.0975912 May 20 04:22:55.147332 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-53652dea-2173-4b30-aecd-47e955e8634e req-37e280a7-b783-4ad9-af6d-06dd856590d7 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0399733 May 20 04:22:55.354454 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9f828869-9c9c-4529-8dad-9443371a98bb req-9ace00dc-a2f0-4232-aa44-5cba66d65dca tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:55.355940 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9f828869-9c9c-4529-8dad-9443371a98bb req-9ace00dc-a2f0-4232-aa44-5cba66d65dca tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 946 time: 0.0835912 May 20 04:22:55.404560 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9f828869-9c9c-4529-8dad-9443371a98bb req-03c7d58d-7ab3-4633-9097-676bb8e54ff5 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0428281 May 20 04:22:55.998413 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-db9f022e-0a18-4e72-af06-6a0f5ecb2457 req-181e6828-3487-4e33-94c4-69085f02f17b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:55.999865 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-db9f022e-0a18-4e72-af06-6a0f5ecb2457 req-181e6828-3487-4e33-94c4-69085f02f17b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1345541 May 20 04:22:56.007029 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-83178b80-ffd1-4d61-b98b-381171d9b4e2 req-9d45a338-f620-4152-b4bd-5b9470d786a8 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:56.008410 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-83178b80-ffd1-4d61-b98b-381171d9b4e2 req-9d45a338-f620-4152-b4bd-5b9470d786a8 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.1127989 May 20 04:22:56.077411 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-db9f022e-0a18-4e72-af06-6a0f5ecb2457 req-39b8ad96-fb92-4c4e-8d14-d9f8a486e93e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0680623 May 20 04:22:56.081830 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-83178b80-ffd1-4d61-b98b-381171d9b4e2 req-2d5f628f-0fba-496a-8ab1-02f0a7ceaee5 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0568335 May 20 04:22:56.330296 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f524e669-9134-4a3b-8c5d-d227e4d83c27 req-493cb836-62a3-474d-9c7f-795716fc9649 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:56.331661 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f524e669-9134-4a3b-8c5d-d227e4d83c27 req-493cb836-62a3-474d-9c7f-795716fc9649 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 945 time: 0.0811689 May 20 04:22:56.378934 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f524e669-9134-4a3b-8c5d-d227e4d83c27 req-3948a4f9-f5d4-47e5-bcca-894f22fc1a33 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0396903 May 20 04:22:56.616527 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c307ffa3-6b55-47c8-9cf0-8e7c46c15cf5 req-fc2e67c7-b5e9-4cb6-803e-17dfdb7650b6 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:56.617873 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c307ffa3-6b55-47c8-9cf0-8e7c46c15cf5 req-fc2e67c7-b5e9-4cb6-803e-17dfdb7650b6 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 946 time: 0.0989676 May 20 04:22:56.665095 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c307ffa3-6b55-47c8-9cf0-8e7c46c15cf5 req-ee4bbb8b-48a4-4be4-b2bf-fdeae2fc5a06 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0414264 May 20 04:22:57.322347 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7f6b9aa5-407f-4f2a-81d8-b7dd71a3e2e6 req-95e23d7a-aeda-4f81-a51d-8347b237f185 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:57.323835 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7f6b9aa5-407f-4f2a-81d8-b7dd71a3e2e6 req-95e23d7a-aeda-4f81-a51d-8347b237f185 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1357741 May 20 04:22:57.335094 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-56a75aae-8dfc-4f07-ba22-82de095be53b req-cb47ade9-10d0-45fb-a3eb-c19998d4adbd tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:57.336486 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-56a75aae-8dfc-4f07-ba22-82de095be53b req-cb47ade9-10d0-45fb-a3eb-c19998d4adbd tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.1472049 May 20 04:22:57.401404 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7f6b9aa5-407f-4f2a-81d8-b7dd71a3e2e6 req-2ab3b596-a63c-4f13-a19b-719dea9874de tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0639353 May 20 04:22:57.408068 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-56a75aae-8dfc-4f07-ba22-82de095be53b req-2fa4a8cb-165c-449d-8359-0cf12e47ca4d tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0560901 May 20 04:22:57.526109 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device a8b0c56a-171d-40e9-beb7-c7b63fa031b9 up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:22:57.583619 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c42cafef-c8df-464a-9097-e5b6a63ac0ec req-031e36e9-ef83-4299-99c8-0dfb05a7cf20 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:57.585472 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c42cafef-c8df-464a-9097-e5b6a63ac0ec req-031e36e9-ef83-4299-99c8-0dfb05a7cf20 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 945 time: 0.0835011 May 20 04:22:57.602391 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port a8b0c56a-171d-40e9-beb7-c7b63fa031b9 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:22:57.606968 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port a8b0c56a-171d-40e9-beb7-c7b63fa031b9 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:22:57.607373 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:57.630607 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c42cafef-c8df-464a-9097-e5b6a63ac0ec req-860b1c23-dddb-4c82-afb5-571c68ae5172 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0394020 May 20 04:22:57.725930 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:57.752426 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '0d234180-40a1-4229-990e-f796674056ce', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'a8b0c56a-171d-40e9-beb7-c7b63fa031b9'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:22:57.763188 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port a8b0c56a-171d-40e9-beb7-c7b63fa031b9, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=a8b0c56a-171d-40e9-beb7-c7b63fa031b9,segment=NetworkSegment(0032d9a3-07bf-489f-bb9d-03ab3ff14f98),segment_id=0032d9a3-07bf-489f-bb9d-03ab3ff14f98)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:22:57.768228 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:22:57.770364 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:22:57.799838 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '0d234180-40a1-4229-990e-f796674056ce', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'a8b0c56a-171d-40e9-beb7-c7b63fa031b9', 'code': 200} May 20 04:22:57.835956 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=a8b0c56a-171d-40e9-beb7-c7b63fa031b9,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:22:57.839129 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:22:57.860005 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-882221f8-5932-4613-80b9-be7072e602bb req-4fda5bd4-dd55-4f6c-8cb3-67ded625c2fc tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:57.861460 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-882221f8-5932-4613-80b9-be7072e602bb req-4fda5bd4-dd55-4f6c-8cb3-67ded625c2fc tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 946 time: 0.0896270 May 20 04:22:57.908357 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-882221f8-5932-4613-80b9-be7072e602bb req-91d1d149-6d21-4ec4-8d6d-abbddf8e9050 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0409670 May 20 04:22:58.646964 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-46d1f3bd-104d-4c9a-9e87-05f57d54ea9d req-b82c5b58-ba28-48ec-bd7f-f40599c9bb2c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:58.648558 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-46d1f3bd-104d-4c9a-9e87-05f57d54ea9d req-b82c5b58-ba28-48ec-bd7f-f40599c9bb2c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1370769 May 20 04:22:58.660408 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-053b33a8-59f6-4ac8-ad68-53a2506a287a req-cbe83838-3d29-4166-821f-5770918ca5a3 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:58.661822 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-053b33a8-59f6-4ac8-ad68-53a2506a287a req-cbe83838-3d29-4166-821f-5770918ca5a3 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.1201091 May 20 04:22:58.729627 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-46d1f3bd-104d-4c9a-9e87-05f57d54ea9d req-1e7dd16c-4af8-415c-8dbf-f1f500656a54 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0668643 May 20 04:22:58.733879 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-053b33a8-59f6-4ac8-ad68-53a2506a287a req-3502ddd3-e976-4d0d-a58f-15605c9dc13b tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0582297 May 20 04:22:58.914994 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8a07f511-a326-463e-a5f0-22bcb2793d02 req-2d27f858-594d-424d-87e0-53e419e4106d tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:58.916577 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8a07f511-a326-463e-a5f0-22bcb2793d02 req-2d27f858-594d-424d-87e0-53e419e4106d tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0862610 May 20 04:22:58.962287 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8a07f511-a326-463e-a5f0-22bcb2793d02 req-d7eb3bdb-3ad1-4d95-9770-793486d37854 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0394788 May 20 04:22:59.089235 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-70f30eff-85e4-402a-848c-2f96c4991cb6 req-f8411d70-216c-491e-9455-ccf5ab2e6da1 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:22:59.090706 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-70f30eff-85e4-402a-848c-2f96c4991cb6 req-f8411d70-216c-491e-9455-ccf5ab2e6da1 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 946 time: 0.0789144 May 20 04:22:59.136224 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-70f30eff-85e4-402a-848c-2f96c4991cb6 req-a74f6b6c-2233-44e8-b70a-1f42b775383a tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0394921 May 20 04:23:00.036441 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b743fc8e-6ac3-432e-b755-12b6259e7bc4 req-f1ee0814-9ab3-4d5b-9322-b69caa28c27b tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:00.037994 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b743fc8e-6ac3-432e-b755-12b6259e7bc4 req-f1ee0814-9ab3-4d5b-9322-b69caa28c27b tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.1942387 May 20 04:23:00.045765 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-25672d39-3a23-4a62-9e95-fc8f393843f3 req-7b7b7bcf-6051-40bf-bebc-f8b3f2f2b27d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:00.047058 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-25672d39-3a23-4a62-9e95-fc8f393843f3 req-7b7b7bcf-6051-40bf-bebc-f8b3f2f2b27d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1172974 May 20 04:23:00.111574 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b743fc8e-6ac3-432e-b755-12b6259e7bc4 req-dac9fa8e-9ea2-4f51-a2d0-13dc7bec7e8e tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0635481 May 20 04:23:00.118811 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-25672d39-3a23-4a62-9e95-fc8f393843f3 req-4c42f68d-9961-4905-a205-9bb3d28be1d5 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0580316 May 20 04:23:00.354831 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e5f041c6-d257-45e3-b5b5-e8b281fd9a7e req-9a8268c1-ffe0-4f37-8c33-35238907c8a1 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:00.356281 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e5f041c6-d257-45e3-b5b5-e8b281fd9a7e req-9a8268c1-ffe0-4f37-8c33-35238907c8a1 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.1507735 May 20 04:23:00.364357 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4e64829d-559f-4f14-8b9a-179f497b6827 req-9ff6b131-fdc8-450e-b947-c060a9d6dc89 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:00.365687 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4e64829d-559f-4f14-8b9a-179f497b6827 req-9ff6b131-fdc8-450e-b947-c060a9d6dc89 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 946 time: 0.1189673 May 20 04:23:00.433034 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e5f041c6-d257-45e3-b5b5-e8b281fd9a7e req-57cdd91d-9228-4816-897e-1c28e91c158c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0664058 May 20 04:23:00.437349 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4e64829d-559f-4f14-8b9a-179f497b6827 req-04973b15-3525-4091-9a94-48cf116b00fb tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0579710 May 20 04:23:00.538585 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f440ff2c-c6bf-4d54-88f9-200c07582ff6 req-b032166d-d75e-4a1f-bf52-744e63a3015f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:00.540498 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f440ff2c-c6bf-4d54-88f9-200c07582ff6 req-b032166d-d75e-4a1f-bf52-744e63a3015f service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 1099 time: 0.0727377 May 20 04:23:00.592587 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f440ff2c-c6bf-4d54-88f9-200c07582ff6 req-74e4969a-8007-4cd9-b3b9-c57045232c95 service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0415320 May 20 04:23:01.361419 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6d56ffcb-741b-4f7a-9a7e-35681fabcfbd req-a2a750cc-a486-4517-a11d-46c0e28a3453 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:01.362847 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6d56ffcb-741b-4f7a-9a7e-35681fabcfbd req-a2a750cc-a486-4517-a11d-46c0e28a3453 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.1408806 May 20 04:23:01.374320 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bf464232-ab9d-49d9-8a45-88c2909b640c req-6febe2e8-1c32-4a47-aca7-2b0ac3219f24 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:01.375721 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bf464232-ab9d-49d9-8a45-88c2909b640c req-6febe2e8-1c32-4a47-aca7-2b0ac3219f24 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1190734 May 20 04:23:01.442119 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6d56ffcb-741b-4f7a-9a7e-35681fabcfbd req-e297c0c7-5e6c-4f47-8b9d-5fb724f43f1f tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0654817 May 20 04:23:01.448899 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bf464232-ab9d-49d9-8a45-88c2909b640c req-44696eae-43d2-400f-8264-eb984bf3fc99 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0598288 May 20 04:23:01.525787 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 706277c6-cd8c-4146-ab91-ed9504446690 up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:23:01.596941 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port 706277c6-cd8c-4146-ab91-ed9504446690 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:23:01.601547 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port 706277c6-cd8c-4146-ab91-ed9504446690 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:23:01.602004 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:01.696792 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e62b8aa1-1cf0-40ad-92f2-7aee6b5367de req-130443d3-843a-4e01-954d-09125dc1db19 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:01.698597 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e62b8aa1-1cf0-40ad-92f2-7aee6b5367de req-130443d3-843a-4e01-954d-09125dc1db19 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.1505566 May 20 04:23:01.707536 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5e2390f7-e854-4172-bc92-949c6c2fb27f req-cb9be973-4e03-41bc-a5d6-8449c80c6c5c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:01.709364 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5e2390f7-e854-4172-bc92-949c6c2fb27f req-cb9be973-4e03-41bc-a5d6-8449c80c6c5c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 946 time: 0.1625764 May 20 04:23:01.717321 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:01.747136 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'bbf6ae7d-8ff5-4b69-adef-47220d6c5c93', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '706277c6-cd8c-4146-ab91-ed9504446690'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:23:01.757095 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 706277c6-cd8c-4146-ab91-ed9504446690, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=706277c6-cd8c-4146-ab91-ed9504446690,segment=NetworkSegment(0032d9a3-07bf-489f-bb9d-03ab3ff14f98),segment_id=0032d9a3-07bf-489f-bb9d-03ab3ff14f98)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:23:01.761690 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:01.764534 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:01.782281 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e62b8aa1-1cf0-40ad-92f2-7aee6b5367de req-92fc1d23-d885-45be-b0b5-33490f46a863 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0718985 May 20 04:23:01.791923 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5e2390f7-e854-4172-bc92-949c6c2fb27f req-321387aa-c8b1-4d52-aedd-8496f71784a2 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0581102 May 20 04:23:01.839732 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'bbf6ae7d-8ff5-4b69-adef-47220d6c5c93', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '706277c6-cd8c-4146-ab91-ed9504446690', 'code': 200} May 20 04:23:01.845375 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=706277c6-cd8c-4146-ab91-ed9504446690,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:01.848622 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:02.159552 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d1f9236f-9156-4172-9e41-d2ee2ae0183e req-00cac780-69fe-47ca-99bb-bb27714d409b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:02.161792 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-d1f9236f-9156-4172-9e41-d2ee2ae0183e req-00cac780-69fe-47ca-99bb-bb27714d409b service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 1098 time: 0.0885725 May 20 04:23:02.247957 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-d1f9236f-9156-4172-9e41-d2ee2ae0183e req-76cd9135-504b-4e07-803b-1becdbb3a7de service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0656688 May 20 04:23:02.709669 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ffd99260-00b0-4c25-94bc-bc610f15a738 req-8365731d-e244-44b6-94bb-c707c00891f5 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:02.711109 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ffd99260-00b0-4c25-94bc-bc610f15a738 req-8365731d-e244-44b6-94bb-c707c00891f5 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.1534746 May 20 04:23:02.723688 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7f249b54-2acd-4b2c-809a-c1473e375c69 req-8261ca7a-c3ad-4115-a4bb-6a51dcd37594 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:02.725213 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7f249b54-2acd-4b2c-809a-c1473e375c69 req-8261ca7a-c3ad-4115-a4bb-6a51dcd37594 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1291847 May 20 04:23:02.796787 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ffd99260-00b0-4c25-94bc-bc610f15a738 req-26d01000-7c40-409e-90bd-82896fc08bd1 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0706577 May 20 04:23:02.804968 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7f249b54-2acd-4b2c-809a-c1473e375c69 req-21a0840f-e630-4289-86e0-cf63c244af13 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0643582 May 20 04:23:03.047364 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-16e66e08-094b-4c8c-a95c-f5d5cfa4f2a5 req-3ad15808-9c73-4e42-ad1e-f12168095899 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:03.048805 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-16e66e08-094b-4c8c-a95c-f5d5cfa4f2a5 req-3ad15808-9c73-4e42-ad1e-f12168095899 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.1472521 May 20 04:23:03.056902 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b12eaeed-8fac-4fcb-b292-d06ccc5bbbf3 req-1ab203d5-f78e-48b8-b2ff-32e4b2ec9511 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:03.058234 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b12eaeed-8fac-4fcb-b292-d06ccc5bbbf3 req-1ab203d5-f78e-48b8-b2ff-32e4b2ec9511 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1112783 May 20 04:23:03.067051 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 34856) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:03.072707 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-ee342260-58b0-44fd-bc5a-c0bede53c376 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Request body: {'floatingip': {'port_id': '535d5637-ae66-43cb-bd16-3543bd3b7a8b'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:23:03.107645 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-16e66e08-094b-4c8c-a95c-f5d5cfa4f2a5 req-608bb2a7-4b69-43d8-9736-cc969539d679 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0480103 May 20 04:23:03.171746 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ee342260-58b0-44fd-bc5a-c0bede53c376 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for floatingip, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:03.521536 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-de7a0c3f-82c7-4ca8-adb1-d0fce1a58400 req-ab226c63-86fa-4539-ad51-d4b3fa702be4 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.2864347 May 20 04:23:03.908552 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-de7a0c3f-82c7-4ca8-adb1-d0fce1a58400 req-230dd5c3-df42-4583-9349-bc06c587cafc service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:03.909990 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-de7a0c3f-82c7-4ca8-adb1-d0fce1a58400 req-230dd5c3-df42-4583-9349-bc06c587cafc service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=51fcbc78284541db832e4780e76158af&device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 1099 time: 0.2952638 May 20 04:23:04.028538 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b2e26cf6-f2ce-4906-b612-107f270c55f1 req-d04c27f3-e19a-441e-8166-66a00ae9db02 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:04.030235 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b2e26cf6-f2ce-4906-b612-107f270c55f1 req-d04c27f3-e19a-441e-8166-66a00ae9db02 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.0959201 May 20 04:23:04.081103 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b2e26cf6-f2ce-4906-b612-107f270c55f1 req-75ffd2d7-abb5-489e-a1d0-f47ba18feee2 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0414884 May 20 04:23:04.092553 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.db.resource_extend [None req-ee342260-58b0-44fd-bc5a-c0bede53c376 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] It took 0.14 seconds to run function 'neutron_lib.db.resource_extend.apply_funcs' {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py:386}} May 20 04:23:04.207853 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ee342260-58b0-44fd-bc5a-c0bede53c376 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for floatingip, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:04.214417 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ee342260-58b0-44fd-bc5a-c0bede53c376 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-8003251'] for floatingip, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:04.307661 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-de7a0c3f-82c7-4ca8-adb1-d0fce1a58400 req-95c058af-7bf0-4e3f-954c-94618e124d55 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:04.308800 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e79bb113-635b-4f9e-ae3d-a1884e8c29b1 req-473d5f47-c69d-47ed-8b5f-e0594f162592 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:04.309201 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-de7a0c3f-82c7-4ca8-adb1-d0fce1a58400 req-95c058af-7bf0-4e3f-954c-94618e124d55 service neutron] 10.222.0.74 "GET /v2.0/networks?id=54fc8d38-193b-49c4-b873-c5353723791f HTTP/1.1" status: 200 len: 916 time: 0.3194978 May 20 04:23:04.310491 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e79bb113-635b-4f9e-ae3d-a1884e8c29b1 req-473d5f47-c69d-47ed-8b5f-e0594f162592 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0858462 May 20 04:23:04.358741 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e79bb113-635b-4f9e-ae3d-a1884e8c29b1 req-a7f0f7f9-609a-434f-ad4a-2a7ea36f5dba tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0415833 May 20 04:23:04.405413 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.db.l3_db [None req-ee342260-58b0-44fd-bc5a-c0bede53c376 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Floating IP 2991a3a7-ad3f-4bb3-a8ca-e45809c0383d associated. External IP: 172.27.0.153, port: 535d5637-ae66-43cb-bd16-3543bd3b7a8b. May 20 04:23:04.890072 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-ee342260-58b0-44fd-bc5a-c0bede53c376 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91644) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:23:04.906372 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-49090f65-5cb1-4c51-9cc4-faff1500350b req-d770da7e-195f-4e84-bc03-4b7b11ef5bbe tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:04.907833 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-49090f65-5cb1-4c51-9cc4-faff1500350b req-d770da7e-195f-4e84-bc03-4b7b11ef5bbe tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0805047 May 20 04:23:04.952135 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-49090f65-5cb1-4c51-9cc4-faff1500350b req-9a298b31-0c9f-480a-baf5-316ba171475b tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0383000 May 20 04:23:05.070490 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-ee342260-58b0-44fd-bc5a-c0bede53c376 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:23:05.084102 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-61bf57d9-7a9a-471c-9a43-a0022a859b6e None None] Sync routers for ids ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'] in dv-cinder-782405-9-iscsi {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:23:05.092768 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ee342260-58b0-44fd-bc5a-c0bede53c376 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:05.093880 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-61bf57d9-7a9a-471c-9a43-a0022a859b6e None None] Routers {'5a3f69b9-fd58-499f-bec9-75215cd7f8fc'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:23:05.094480 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-61bf57d9-7a9a-471c-9a43-a0022a859b6e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:22:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:23:05.147359 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-ee342260-58b0-44fd-bc5a-c0bede53c376 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "PUT /v2.0/floatingips/2991a3a7-ad3f-4bb3-a8ca-e45809c0383d HTTP/1.1" status: 200 len: 991 time: 2.0788198 May 20 04:23:05.147831 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'c2ca950f-78ff-4e44-9231-e8e19f4aee5d', 'tag': '535d5637-ae66-43cb-bd16-3543bd3b7a8b'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:23:05.273497 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-61bf57d9-7a9a-471c-9a43-a0022a859b6e None None] Set router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:23:05.533314 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:05.623264 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-61bf57d9-7a9a-471c-9a43-a0022a859b6e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port': {'id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:e4:9a:7c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:39Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': '2991a3a7-ad3f-4bb3-a8ca-e45809c0383d', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'floating_ip_address': '172.27.0.153', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'port_id': '535d5637-ae66-43cb-bd16-3543bd3b7a8b', 'fixed_ip_address': '172.20.0.6', 'status': 'DOWN', 'standard_attr_id': 121, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'mac_address': 'fa:16:3e:ea:90:d5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ca950f-78ff-4e44-9231-e8e19f4aee5d', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:22:00Z', 'updated_at': '2021-05-20T04:23:03Z', 'revision_number': 1, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'ac9495b7-5868-4104-85c9-3567f1350467', 'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'mac_address': 'fa:16:3e:12:5b:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_interface', 'standard_attr_id': 82, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:45Z', 'updated_at': '2021-05-20T04:21:55Z', 'revision_number': 5, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'subnets': [{'id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:23:05.627134 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-61bf57d9-7a9a-471c-9a43-a0022a859b6e None None] Checking router: 5a3f69b9-fd58-499f-bec9-75215cd7f8fc for host: dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:23:05.635685 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-61bf57d9-7a9a-471c-9a43-a0022a859b6e None None] Routers {'5a3f69b9-fd58-499f-bec9-75215cd7f8fc'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:23:05.636271 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-61bf57d9-7a9a-471c-9a43-a0022a859b6e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:22:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:23:05.814231 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-61bf57d9-7a9a-471c-9a43-a0022a859b6e None None] Set router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:23:06.194633 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-61bf57d9-7a9a-471c-9a43-a0022a859b6e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port': {'id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:e4:9a:7c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:39Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': '2991a3a7-ad3f-4bb3-a8ca-e45809c0383d', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'floating_ip_address': '172.27.0.153', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'port_id': '535d5637-ae66-43cb-bd16-3543bd3b7a8b', 'fixed_ip_address': '172.20.0.6', 'status': 'DOWN', 'standard_attr_id': 121, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'mac_address': 'fa:16:3e:ea:90:d5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ca950f-78ff-4e44-9231-e8e19f4aee5d', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:22:00Z', 'updated_at': '2021-05-20T04:23:03Z', 'revision_number': 1, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'ac9495b7-5868-4104-85c9-3567f1350467', 'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'mac_address': 'fa:16:3e:12:5b:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_interface', 'standard_attr_id': 82, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:45Z', 'updated_at': '2021-05-20T04:21:55Z', 'revision_number': 5, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'subnets': [{'id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:23:06.198743 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-61bf57d9-7a9a-471c-9a43-a0022a859b6e None None] The sync data for ids ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'] in dv-cinder-782405-9-iscsi is: [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port': {'id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:e4:9a:7c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:39Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': '2991a3a7-ad3f-4bb3-a8ca-e45809c0383d', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'floating_ip_address': '172.27.0.153', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'port_id': '535d5637-ae66-43cb-bd16-3543bd3b7a8b', 'fixed_ip_address': '172.20.0.6', 'status': 'DOWN', 'standard_attr_id': 121, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'mac_address': 'fa:16:3e:ea:90:d5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'c2ca950f-78ff-4e44-9231-e8e19f4aee5d', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:22:00Z', 'updated_at': '2021-05-20T04:23:03Z', 'revision_number': 1, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'ac9495b7-5868-4104-85c9-3567f1350467', 'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'mac_address': 'fa:16:3e:12:5b:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_interface', 'standard_attr_id': 82, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:45Z', 'updated_at': '2021-05-20T04:21:55Z', 'revision_number': 5, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'subnets': [{'id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:23:06.395656 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-60651fdb-3981-4ee0-934b-50a541eb8b06 None None] New status for floating IP 2991a3a7-ad3f-4bb3-a8ca-e45809c0383d: ACTIVE {{(pid=91645) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 20 04:23:09.657187 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-de7a0c3f-82c7-4ca8-adb1-d0fce1a58400 req-c56b66d7-1ae6-4c57-b012-d0b1f901cc22 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.14&port_id=849868be-af51-4967-9a6c-93d23cecb450 HTTP/1.1" status: 200 len: 217 time: 0.0694072 May 20 04:23:11.728560 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-de7a0c3f-82c7-4ca8-adb1-d0fce1a58400 req-a0522535-9d97-4751-9471-70acf3eaf578 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:11.730002 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-de7a0c3f-82c7-4ca8-adb1-d0fce1a58400 req-a0522535-9d97-4751-9471-70acf3eaf578 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=0f918085-8d33-4e36-a59e-7f270a2b1874 HTTP/1.1" status: 200 len: 855 time: 0.0816126 May 20 04:23:11.744204 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'c2ca950f-78ff-4e44-9231-e8e19f4aee5d', 'tag': '535d5637-ae66-43cb-bd16-3543bd3b7a8b', 'status': 'completed', 'code': 200} May 20 04:23:11.747080 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d2271f71-4aab-45ed-a777-3eb0141a4af5 req-e43d94a8-8bb0-4caf-bc1b-381074e5e249 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:11.748563 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-d2271f71-4aab-45ed-a777-3eb0141a4af5 req-e43d94a8-8bb0-4caf-bc1b-381074e5e249 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=bec56d8703cf47f3b7193b7d1f55396d&device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 1099 time: 0.1305697 May 20 04:23:11.791285 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-de7a0c3f-82c7-4ca8-adb1-d0fce1a58400 req-44a6a361-3d3a-4d9f-8434-81bdc944d77d service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=54fc8d38-193b-49c4-b873-c5353723791f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0417805 May 20 04:23:11.868379 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1505631f-51d6-4651-97a8-ed016617377f req-95f58dce-b7dc-4c83-aca6-4e7b1049ee71 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:11.870289 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1505631f-51d6-4651-97a8-ed016617377f req-95f58dce-b7dc-4c83-aca6-4e7b1049ee71 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=9fafdf3a29024ac588e5b24719735677&device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 1098 time: 0.0971074 May 20 04:23:11.938897 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d2271f71-4aab-45ed-a777-3eb0141a4af5 req-50a47ecd-6643-467e-9b26-4546a9155d62 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:11.940443 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-d2271f71-4aab-45ed-a777-3eb0141a4af5 req-50a47ecd-6643-467e-9b26-4546a9155d62 service neutron] 10.222.0.74 "GET /v2.0/networks?id=36b37213-8439-447d-9eba-a78baf7c698e HTTP/1.1" status: 200 len: 920 time: 0.1482708 May 20 04:23:12.033224 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-d2271f71-4aab-45ed-a777-3eb0141a4af5 req-a9d86d0d-76cb-4434-b5c6-0261035cfaa9 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.13&port_id=a8b0c56a-171d-40e9-beb7-c7b63fa031b9 HTTP/1.1" status: 200 len: 217 time: 0.0767372 May 20 04:23:12.161982 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1505631f-51d6-4651-97a8-ed016617377f req-8b170c7e-e4e7-407d-b163-6071b581ec19 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:12.163504 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1505631f-51d6-4651-97a8-ed016617377f req-8b170c7e-e4e7-407d-b163-6071b581ec19 service neutron] 10.222.0.74 "GET /v2.0/networks?id=64bcea8a-6d35-406b-a823-fa1185008174 HTTP/1.1" status: 200 len: 918 time: 0.2515876 May 20 04:23:12.169781 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-de7a0c3f-82c7-4ca8-adb1-d0fce1a58400 req-f46a4027-74bd-4a77-b2ff-9d075c789cfa service neutron] 10.222.0.74 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3474503 May 20 04:23:12.174865 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5a997e66-a5b5-4e4b-9d46-b417345ec88e req-c7db5d9f-6917-425f-bb0a-8df2b7b80bfb tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:12.176355 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5a997e66-a5b5-4e4b-9d46-b417345ec88e req-c7db5d9f-6917-425f-bb0a-8df2b7b80bfb tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.3050213 May 20 04:23:12.181609 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d2271f71-4aab-45ed-a777-3eb0141a4af5 req-62a5f528-7b2b-48c7-8794-02b5ef302d6a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:12.183047 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-d2271f71-4aab-45ed-a777-3eb0141a4af5 req-62a5f528-7b2b-48c7-8794-02b5ef302d6a service neutron] 10.222.0.74 "GET /v2.0/subnets?id=b25ae9c8-e11d-45ee-bde2-332cc1538602 HTTP/1.1" status: 200 len: 859 time: 0.0737197 May 20 04:23:12.260022 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-d2271f71-4aab-45ed-a777-3eb0141a4af5 req-9e240e7f-a862-4ec4-bebb-8d473b527bf5 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=36b37213-8439-447d-9eba-a78baf7c698e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0565717 May 20 04:23:12.309632 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5a997e66-a5b5-4e4b-9d46-b417345ec88e req-ec0a9b50-a3d0-47be-af77-f182b532f59d tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0731087 May 20 04:23:12.312694 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1505631f-51d6-4651-97a8-ed016617377f req-e7b6d227-c2d9-4380-86b0-a7cf4f10677e service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=535d5637-ae66-43cb-bd16-3543bd3b7a8b HTTP/1.1" status: 200 len: 1001 time: 0.1296928 May 20 04:23:12.402791 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1505631f-51d6-4651-97a8-ed016617377f req-a0ecc0bf-e667-48f2-b40d-a9ba3b6155f7 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:12.403828 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1505631f-51d6-4651-97a8-ed016617377f req-a0ecc0bf-e667-48f2-b40d-a9ba3b6155f7 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=203a3962-2282-4f51-ac92-c9695b5c0cfe HTTP/1.1" status: 200 len: 856 time: 0.0841563 May 20 04:23:12.482833 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1505631f-51d6-4651-97a8-ed016617377f req-98b0c1b2-f721-47ef-a1a1-3dff68dbca8d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:12.484449 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1505631f-51d6-4651-97a8-ed016617377f req-98b0c1b2-f721-47ef-a1a1-3dff68dbca8d service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=64bcea8a-6d35-406b-a823-fa1185008174&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0708013 May 20 04:23:12.534850 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-de7a0c3f-82c7-4ca8-adb1-d0fce1a58400 req-4f8ba2e1-1c89-44b6-8c88-573e3535cc75 service neutron] 10.222.0.74 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3508012 May 20 04:23:12.565634 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-d2271f71-4aab-45ed-a777-3eb0141a4af5 req-d0b80c68-255d-4289-959b-2c8943ee3e70 service neutron] 10.222.0.74 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2622020 May 20 04:23:12.595689 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-fcd5ca0e-f855-4cc9-b93e-e37b2819659d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:12.637629 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1505631f-51d6-4651-97a8-ed016617377f req-55146662-50b2-4e3d-a449-e818368cd6bd service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1436703 May 20 04:23:12.728420 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-d2271f71-4aab-45ed-a777-3eb0141a4af5 req-40d960b0-bf3f-4b09-a974-8f2e0eaf4663 service neutron] 10.222.0.74 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1429231 May 20 04:23:12.783488 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1505631f-51d6-4651-97a8-ed016617377f req-8f6e33fb-bbd3-41fc-9a5a-db377861aabc service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1332295 May 20 04:23:13.500983 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2e402bf8-d3d1-40d8-81f6-d12e13c7e5f6 req-1bfe5d74-38b1-4cab-a474-ce4ee1092e37 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:13.502427 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2e402bf8-d3d1-40d8-81f6-d12e13c7e5f6 req-1bfe5d74-38b1-4cab-a474-ce4ee1092e37 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0845826 May 20 04:23:13.555995 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2e402bf8-d3d1-40d8-81f6-d12e13c7e5f6 req-23295f36-e1ac-4d1b-b879-af0d69b59c87 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0476913 May 20 04:23:14.737890 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-08d92288-ca39-4c8f-8d86-f3937a39b96a req-a058a9c7-6fa8-4ac4-802d-2c6794b0e731 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:14.739379 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-08d92288-ca39-4c8f-8d86-f3937a39b96a req-a058a9c7-6fa8-4ac4-802d-2c6794b0e731 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0785308 May 20 04:23:14.788208 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-08d92288-ca39-4c8f-8d86-f3937a39b96a req-66ac53f7-92d8-4881-88a1-22ba57841244 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0425093 May 20 04:23:14.960312 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:15.076324 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0412c915-9619-446d-88ed-6fbf708c00a7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:16.117912 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-73ae5493-84b8-4651-9f40-7c602c5c1768 req-e4e69737-9dd5-42c7-89db-bad99f4590bd tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:16.119411 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-73ae5493-84b8-4651-9f40-7c602c5c1768 req-e4e69737-9dd5-42c7-89db-bad99f4590bd tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0902677 May 20 04:23:16.163893 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:16.164492 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-73ae5493-84b8-4651-9f40-7c602c5c1768 req-8419e142-987f-44ca-b8b2-2b72626955e0 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0386486 May 20 04:23:16.801525 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:23:17.361266 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-24f05faa-9ce6-43a0-8c0f-052e36d6cae6 req-000dbb49-4496-4723-9f25-5eee2820378b tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:17.362738 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-24f05faa-9ce6-43a0-8c0f-052e36d6cae6 req-000dbb49-4496-4723-9f25-5eee2820378b tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0879216 May 20 04:23:17.414187 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-24f05faa-9ce6-43a0-8c0f-052e36d6cae6 req-da31e502-a1ae-43c7-aacf-3c91c27f020b tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0455568 May 20 04:23:17.797343 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:18.619076 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0a9d1bd7-41ac-4d57-a485-9c57934f80dd req-ab36515d-9912-42cc-8ee2-4a260112207b tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:18.620552 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0a9d1bd7-41ac-4d57-a485-9c57934f80dd req-ab36515d-9912-42cc-8ee2-4a260112207b tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0927894 May 20 04:23:18.666015 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0a9d1bd7-41ac-4d57-a485-9c57934f80dd req-c8ad1af7-2f6f-482b-835c-c7e400607cfe tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0396225 May 20 04:23:19.865590 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-080f45f2-c70b-404e-bdcc-b3079acf9181 req-4972c1a9-c81b-491b-9a13-c3902baecdac tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:19.867046 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-080f45f2-c70b-404e-bdcc-b3079acf9181 req-4972c1a9-c81b-491b-9a13-c3902baecdac tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0864024 May 20 04:23:19.911858 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-080f45f2-c70b-404e-bdcc-b3079acf9181 req-efff0959-4c78-4d5b-ae08-71bfb0fee2aa tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0389073 May 20 04:23:21.108274 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6950518f-8668-40c7-a06b-5069f91e9ce8 req-33b1c799-c39b-402c-a8c3-0926ece632fd tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:21.109697 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6950518f-8668-40c7-a06b-5069f91e9ce8 req-33b1c799-c39b-402c-a8c3-0926ece632fd tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0858431 May 20 04:23:21.154766 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6950518f-8668-40c7-a06b-5069f91e9ce8 req-36a078d4-3067-4984-9033-21fa2c9eadc4 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0386925 May 20 04:23:22.338739 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-51b2ac68-2c22-4b92-b165-07e319dc8821 req-d02ee007-7ede-487c-8218-937ee15de983 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:22.340257 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-51b2ac68-2c22-4b92-b165-07e319dc8821 req-d02ee007-7ede-487c-8218-937ee15de983 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0790231 May 20 04:23:22.384737 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-51b2ac68-2c22-4b92-b165-07e319dc8821 req-bf5132fa-b546-445e-81a8-5caf7d1a61a0 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0384014 May 20 04:23:23.573687 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cd9ac591-7aa9-4a7a-92bc-135afaedf853 req-6e80d512-d8c8-4cd2-8d6e-b03799518f5e tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:23.575004 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cd9ac591-7aa9-4a7a-92bc-135afaedf853 req-6e80d512-d8c8-4cd2-8d6e-b03799518f5e tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0833755 May 20 04:23:23.620352 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cd9ac591-7aa9-4a7a-92bc-135afaedf853 req-22ee53ed-833c-456e-b10a-d46e42bf2e5e tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0394218 May 20 04:23:23.933060 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 35202) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:24.055670 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-ebce0eea-a0e7-4c14-bfa7-fe9b9900d07f admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:24.071569 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-ebce0eea-a0e7-4c14-bfa7-fe9b9900d07f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:23:24.075715 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-ebce0eea-a0e7-4c14-bfa7-fe9b9900d07f admin admin] Usage tracker for resource:security_group_rule and tenant:42c4d0f19ee04a4c927506ad635cf916 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:23:24.078747 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-ebce0eea-a0e7-4c14-bfa7-fe9b9900d07f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:23:24.085015 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-ebce0eea-a0e7-4c14-bfa7-fe9b9900d07f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:23:24.165497 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-ebce0eea-a0e7-4c14-bfa7-fe9b9900d07f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:24.173716 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-ebce0eea-a0e7-4c14-bfa7-fe9b9900d07f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:24.181873 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-48e57dc6-e1e3-4200-a9d8-67faeacdb318 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:24.222906 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-ebce0eea-a0e7-4c14-bfa7-fe9b9900d07f admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=42c4d0f19ee04a4c927506ad635cf916&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2887883 May 20 04:23:24.224567 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ebce0eea-a0e7-4c14-bfa7-fe9b9900d07f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=6b621eba-1ab6-449c-841f-57b4b32cf804,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:24.225669 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 35208) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:24.227671 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-ebce0eea-a0e7-4c14-bfa7-fe9b9900d07f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:24.285203 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-31ab0065-c8e8-4888-a4ca-f153dfe60850 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:24.310640 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-31ab0065-c8e8-4888-a4ca-f153dfe60850 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:24.364440 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-31ab0065-c8e8-4888-a4ca-f153dfe60850 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:24.368174 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-b1e4a4a8-2e70-4802-8d88-9544e2c048d2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:24.379121 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-31ab0065-c8e8-4888-a4ca-f153dfe60850 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=6b621eba-1ab6-449c-841f-57b4b32cf804,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:24.381586 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-31ab0065-c8e8-4888-a4ca-f153dfe60850 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.013s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:24.387656 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-31ab0065-c8e8-4888-a4ca-f153dfe60850 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:24.388752 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-31ab0065-c8e8-4888-a4ca-f153dfe60850 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/6b621eba-1ab6-449c-841f-57b4b32cf804 HTTP/1.1" status: 204 len: 168 time: 0.1620526 May 20 04:23:24.689371 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 35228) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:24.798274 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-707e6e81-8983-4099-8011-96084158973e admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:24.825633 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-707e6e81-8983-4099-8011-96084158973e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:23:24.829147 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-707e6e81-8983-4099-8011-96084158973e admin admin] Usage tracker for resource:security_group_rule and tenant:b0196cdfd68d456399655acb503e895b is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:23:24.833354 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-707e6e81-8983-4099-8011-96084158973e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:23:24.845895 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-707e6e81-8983-4099-8011-96084158973e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:23:24.902289 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6068e982-1736-4bee-b40f-1b0e7b86f49a req-e59f618e-6270-4387-b3a1-a0728eee3335 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:24.903779 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6068e982-1736-4bee-b40f-1b0e7b86f49a req-e59f618e-6270-4387-b3a1-a0728eee3335 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.1399462 May 20 04:23:24.989285 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6068e982-1736-4bee-b40f-1b0e7b86f49a req-f34754d1-0208-4e50-b90e-42fd1859cb2a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0740743 May 20 04:23:25.016437 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-707e6e81-8983-4099-8011-96084158973e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:25.020231 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-707e6e81-8983-4099-8011-96084158973e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:25.028637 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-7711194d-933c-413d-b7b2-59b89552c2c9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:25.068540 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-707e6e81-8983-4099-8011-96084158973e admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=b0196cdfd68d456399655acb503e895b&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3780310 May 20 04:23:25.069683 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-707e6e81-8983-4099-8011-96084158973e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=6ac17fea-5efc-4f6d-a2fa-8ad05d703869,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:25.071194 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 35244) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:25.072720 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-707e6e81-8983-4099-8011-96084158973e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:25.134529 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8d161969-b827-4f57-ba13-0474ce3ebbb9 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:25.162283 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8d161969-b827-4f57-ba13-0474ce3ebbb9 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:25.219878 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8d161969-b827-4f57-ba13-0474ce3ebbb9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:25.223653 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:25.236483 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8d161969-b827-4f57-ba13-0474ce3ebbb9 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=6ac17fea-5efc-4f6d-a2fa-8ad05d703869,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:25.239499 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-8d161969-b827-4f57-ba13-0474ce3ebbb9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:25.241554 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8d161969-b827-4f57-ba13-0474ce3ebbb9 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:25.242640 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-8d161969-b827-4f57-ba13-0474ce3ebbb9 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/6ac17fea-5efc-4f6d-a2fa-8ad05d703869 HTTP/1.1" status: 204 len: 168 time: 0.1698434 May 20 04:23:26.289547 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c2fc5d51-5189-4f1c-a259-1ded9fd9594a req-8fbd1ea4-d086-4a08-87e9-5f3116ed6ec6 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:26.291012 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c2fc5d51-5189-4f1c-a259-1ded9fd9594a req-8fbd1ea4-d086-4a08-87e9-5f3116ed6ec6 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.1090386 May 20 04:23:26.343929 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c2fc5d51-5189-4f1c-a259-1ded9fd9594a req-4e78fa31-a777-46a8-915d-602baca72630 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0468519 May 20 04:23:26.578420 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 35314) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:26.817752 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1e5ac10d-e8d3-4ecb-be30-1db7c1f2857d admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:26.832838 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-1e5ac10d-e8d3-4ecb-be30-1db7c1f2857d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:23:26.836921 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-1e5ac10d-e8d3-4ecb-be30-1db7c1f2857d admin admin] Usage tracker for resource:security_group_rule and tenant:a9889b3a86c649a589b05df9f9f76496 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:23:26.840011 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-1e5ac10d-e8d3-4ecb-be30-1db7c1f2857d admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:23:26.846212 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-1e5ac10d-e8d3-4ecb-be30-1db7c1f2857d admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:23:26.928238 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1e5ac10d-e8d3-4ecb-be30-1db7c1f2857d admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:26.932585 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1e5ac10d-e8d3-4ecb-be30-1db7c1f2857d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:26.940831 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-803f2476-d3a1-4a4c-a276-a15b23c2539c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:26.979738 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-1e5ac10d-e8d3-4ecb-be30-1db7c1f2857d admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=a9889b3a86c649a589b05df9f9f76496&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4001119 May 20 04:23:26.980732 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1e5ac10d-e8d3-4ecb-be30-1db7c1f2857d admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e10b7161-9ea7-44c4-94e4-f0fa73eca6d3,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:26.982482 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 35332) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:26.983878 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-1e5ac10d-e8d3-4ecb-be30-1db7c1f2857d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:27.042178 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b03d7900-1588-488b-9abd-4c665f8c0427 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:27.068303 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b03d7900-1588-488b-9abd-4c665f8c0427 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:27.122035 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b03d7900-1588-488b-9abd-4c665f8c0427 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:27.130425 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:27.144065 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b03d7900-1588-488b-9abd-4c665f8c0427 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e10b7161-9ea7-44c4-94e4-f0fa73eca6d3,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:27.148827 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-b03d7900-1588-488b-9abd-4c665f8c0427 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:27.165164 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b03d7900-1588-488b-9abd-4c665f8c0427 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:27.170730 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-b03d7900-1588-488b-9abd-4c665f8c0427 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/e10b7161-9ea7-44c4-94e4-f0fa73eca6d3 HTTP/1.1" status: 204 len: 168 time: 0.1872294 May 20 04:23:27.547754 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c628222c-3d44-4182-b562-86a2b15b79a3 req-badca1e6-89a7-45d3-aa50-9b094940a63f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:27.549291 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c628222c-3d44-4182-b562-86a2b15b79a3 req-badca1e6-89a7-45d3-aa50-9b094940a63f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0882914 May 20 04:23:27.594319 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c628222c-3d44-4182-b562-86a2b15b79a3 req-04d1156e-b165-496c-b684-7a7b64cfe21f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0390275 May 20 04:23:28.793382 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8ff0b91b-816c-48c9-a86a-4b49752e33ed req-f3592936-2d74-413c-acc0-891eb4971f2a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:28.794851 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8ff0b91b-816c-48c9-a86a-4b49752e33ed req-f3592936-2d74-413c-acc0-891eb4971f2a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0919733 May 20 04:23:28.844150 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8ff0b91b-816c-48c9-a86a-4b49752e33ed req-ea535c8c-cee3-42e6-91d4-9446ee62072b tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0431211 May 20 04:23:30.036664 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b6f6c16b-a56f-4586-91aa-a751f8fc1252 req-80215f9c-88d5-40ce-b199-a347f1c94a92 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:30.038210 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b6f6c16b-a56f-4586-91aa-a751f8fc1252 req-80215f9c-88d5-40ce-b199-a347f1c94a92 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0827994 May 20 04:23:30.083351 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b6f6c16b-a56f-4586-91aa-a751f8fc1252 req-c721d98f-b732-477d-adfd-444944d95191 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0393045 May 20 04:23:31.279693 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f6f1744c-5c35-48d8-b0ec-bb4764359c78 req-de16af12-be72-49d3-9d4e-eebcc92d5039 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:31.281068 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f6f1744c-5c35-48d8-b0ec-bb4764359c78 req-de16af12-be72-49d3-9d4e-eebcc92d5039 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0911319 May 20 04:23:31.325093 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f6f1744c-5c35-48d8-b0ec-bb4764359c78 req-da003881-86e9-4809-92c5-9ecdf62ab7af tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0383234 May 20 04:23:32.538145 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1a2cfee0-dc16-4a60-8531-c6bc7e30e7fc req-d2d44daa-3657-419e-bd1e-a3a17a8a1137 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:32.539621 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1a2cfee0-dc16-4a60-8531-c6bc7e30e7fc req-d2d44daa-3657-419e-bd1e-a3a17a8a1137 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0821934 May 20 04:23:32.587671 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1a2cfee0-dc16-4a60-8531-c6bc7e30e7fc req-ad863f94-dd12-4864-9cea-8dc16d04622e tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0424066 May 20 04:23:33.795890 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-928cf9f4-0927-4973-8c8b-3a6f7889f9ee tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:33.797320 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-928cf9f4-0927-4973-8c8b-3a6f7889f9ee tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/networks?id=54fc8d38-193b-49c4-b873-c5353723791f HTTP/1.1" status: 200 len: 815 time: 0.1989803 May 20 04:23:33.816804 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-b867c282-1ed1-4ef8-b7f4-353882ccea34 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/quotas/51fcbc78284541db832e4780e76158af HTTP/1.1" status: 200 len: 375 time: 0.0127370 May 20 04:23:33.824779 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b95266cc-1c9b-4ff3-a292-e118273f46c0 req-7deba5be-2dba-4695-8639-0dc8faa2c275 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:33.826142 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b95266cc-1c9b-4ff3-a292-e118273f46c0 req-7deba5be-2dba-4695-8639-0dc8faa2c275 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.1192195 May 20 04:23:33.916602 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b95266cc-1c9b-4ff3-a292-e118273f46c0 req-0d63f055-d7a4-4c58-be9c-aaccf2684994 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0592837 May 20 04:23:33.938123 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-f923857c-eeb6-46a9-a9d5-3b69e0ef6e01 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=51fcbc78284541db832e4780e76158af&fields=id HTTP/1.1" status: 200 len: 255 time: 0.1110871 May 20 04:23:34.068679 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-ae208ba5-a5af-457f-8462-dcec9e4ab27b service neutron] 10.222.0.39 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1232629 May 20 04:23:34.204669 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-67426664-729b-4858-a217-1f1b08fd436d service neutron] 10.222.0.39 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1288135 May 20 04:23:34.383619 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-25c15c59-27e2-43b9-adfb-8684ed1e8649 req-02cf2fe7-9247-4dbf-a5e0-9d95ae2f88c1 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 210 time: 0.0421591 May 20 04:23:35.125440 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c8dfab53-b0fb-472b-b489-22568002ee6f req-956bd87c-5ebd-4549-8753-10dc18b6d3dc tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:35.126916 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c8dfab53-b0fb-472b-b489-22568002ee6f req-956bd87c-5ebd-4549-8753-10dc18b6d3dc tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0948699 May 20 04:23:35.240213 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c8dfab53-b0fb-472b-b489-22568002ee6f req-19400a41-6102-4393-be77-0b1ae0528944 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1023307 May 20 04:23:35.536397 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:35.544490 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-57567374-a0b5-49c4-b0ca-941c385d72e7 req-187f4341-3220-4aea-abd9-72e19ae6ac0e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 210 time: 0.0461030 May 20 04:23:36.433872 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eab41185-37ee-44de-9be7-70586f284b40 req-22da2876-e523-4e40-af2f-d7232e09cd8d tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:36.435243 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-eab41185-37ee-44de-9be7-70586f284b40 req-22da2876-e523-4e40-af2f-d7232e09cd8d tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0864897 May 20 04:23:36.480844 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-eab41185-37ee-44de-9be7-70586f284b40 req-c55329ed-6e70-4031-ae2e-82e701b30b1d tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0393116 May 20 04:23:36.694982 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3e047f40-11b3-4a9c-b56b-8593dd45958e req-96cf40a4-dcf1-44ea-8d6c-0178479e778a tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 210 time: 0.0473959 May 20 04:23:37.693173 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-93a21aae-d54e-4579-a1de-3559f4e27339 req-78ce32af-d875-4d74-9bf8-d374b0bb8914 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:37.694590 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-93a21aae-d54e-4579-a1de-3559f4e27339 req-78ce32af-d875-4d74-9bf8-d374b0bb8914 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0966194 May 20 04:23:37.738818 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-93a21aae-d54e-4579-a1de-3559f4e27339 req-0ad7a87b-8689-418e-b9fe-c6036d32eff2 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0383699 May 20 04:23:37.888150 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-21dcdd62-2e6c-4f57-be3e-5caf54394b27 req-37eba566-ecff-47cf-8e22-ad612ca23c13 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 210 time: 0.0558369 May 20 04:23:38.936576 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-69aea13c-0f1d-4f70-b952-324fe2db5c7d req-5e10e923-3af5-44ea-8246-cd03d66d690a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:38.938043 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-69aea13c-0f1d-4f70-b952-324fe2db5c7d req-5e10e923-3af5-44ea-8246-cd03d66d690a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0820844 May 20 04:23:38.998243 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-69aea13c-0f1d-4f70-b952-324fe2db5c7d req-dfced2e9-56c6-40c3-8a7a-a22c1b775dee tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0542772 May 20 04:23:39.092710 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7bc96fc0-4773-4dac-af6e-87b6caf41221 req-c8ed123b-a6a2-4e61-8347-cfdd3d56243b tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 210 time: 0.0742500 May 20 04:23:39.200290 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-c80a94a4-0cd5-4e92-ba38-291618b0b0f1 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:39.201674 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-c80a94a4-0cd5-4e92-ba38-291618b0b0f1 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.74 "GET /v2.0/networks?id=54fc8d38-193b-49c4-b873-c5353723791f HTTP/1.1" status: 200 len: 815 time: 0.1518080 May 20 04:23:39.220848 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-b9043a63-98af-47f8-a045-7d331dfb621e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Request body: {'port': {'device_id': 'bc7b7a9f-a875-49ec-a1ed-47a7798818e7', 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f', 'admin_state_up': True, 'tenant_id': '51fcbc78284541db832e4780e76158af'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:23:39.354938 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-b9043a63-98af-47f8-a045-7d331dfb621e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:23:39.358662 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-b9043a63-98af-47f8-a045-7d331dfb621e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Usage tracker for resource:port and tenant:51fcbc78284541db832e4780e76158af is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:23:39.361557 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-b9043a63-98af-47f8-a045-7d331dfb621e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Quota usage for port was recalculated. Used quota:1. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:23:39.367568 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-b9043a63-98af-47f8-a045-7d331dfb621e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attempting to reserve 1 items for resource port. Total usage: 1; quota limit: 500; headroom:499 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:23:39.377682 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-b9043a63-98af-47f8-a045-7d331dfb621e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Made reservation on behalf of 51fcbc78284541db832e4780e76158af for: {'port': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:23:39.381179 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-b9043a63-98af-47f8-a045-7d331dfb621e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:39.563962 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-b9043a63-98af-47f8-a045-7d331dfb621e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Allocated IP 172.20.0.3 (54fc8d38-193b-49c4-b873-c5353723791f/0f918085-8d33-4e36-a59e-7f270a2b1874/643e5696-1a48-46bc-9a75-8b39f176b597) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:23:39.767343 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-b9043a63-98af-47f8-a045-7d331dfb621e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:39.799186 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-b9043a63-98af-47f8-a045-7d331dfb621e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:39.806485 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:39.871533 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-b9043a63-98af-47f8-a045-7d331dfb621e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Network 54fc8d38-193b-49c4-b873-c5353723791f is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:23:39.874611 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-b9043a63-98af-47f8-a045-7d331dfb621e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:23:39.878853 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-b9043a63-98af-47f8-a045-7d331dfb621e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:23:39.884545 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-b9043a63-98af-47f8-a045-7d331dfb621e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:39.885156 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-b9043a63-98af-47f8-a045-7d331dfb621e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:39.901465 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b9043a63-98af-47f8-a045-7d331dfb621e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Pushing event updated for resources: {'Port': ['ID=643e5696-1a48-46bc-9a75-8b39f176b597,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:39.904282 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-b9043a63-98af-47f8-a045-7d331dfb621e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.098s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:39.913311 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-b9043a63-98af-47f8-a045-7d331dfb621e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 0.6965601 May 20 04:23:40.205557 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bcae637f-96cd-46e3-a7ed-73dde46bcb5f req-755086a6-7b9d-497a-9671-8b400454b109 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:40.211700 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bcae637f-96cd-46e3-a7ed-73dde46bcb5f req-755086a6-7b9d-497a-9671-8b400454b109 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.1062489 May 20 04:23:40.315868 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bcae637f-96cd-46e3-a7ed-73dde46bcb5f req-224fa52d-7442-41bc-a969-78bd6f42ce33 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1030080 May 20 04:23:40.343940 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e6c87630-e2be-49d4-9723-819b568a01a9 req-ff4749f7-b075-4425-aba9-e921672ba0f7 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:40.344640 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-e6c87630-e2be-49d4-9723-819b568a01a9 req-ff4749f7-b075-4425-aba9-e921672ba0f7 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Synchronizing usage tracker for tenant:51fcbc78284541db832e4780e76158af on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:23:40.361448 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-e6c87630-e2be-49d4-9723-819b568a01a9 req-ff4749f7-b075-4425-aba9-e921672ba0f7 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Unset dirty status for tenant:51fcbc78284541db832e4780e76158af on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:23:40.362631 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e6c87630-e2be-49d4-9723-819b568a01a9 req-ff4749f7-b075-4425-aba9-e921672ba0f7 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 931 time: 0.1329455 May 20 04:23:40.406707 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e6c87630-e2be-49d4-9723-819b568a01a9 req-ac087d2c-de8a-43d9-8737-6831519bb0af tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0382864 May 20 04:23:41.497332 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7927b303-8ac0-44b0-8988-cdd9128a7cbc req-0158f2b8-ceef-428b-b078-e52accfe6c00 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:41.498772 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7927b303-8ac0-44b0-8988-cdd9128a7cbc req-0158f2b8-ceef-428b-b078-e52accfe6c00 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0808415 May 20 04:23:41.545843 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7927b303-8ac0-44b0-8988-cdd9128a7cbc req-3dc99546-0a17-44fd-a90a-73414332b47b tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0410538 May 20 04:23:41.684632 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['849868be-af51-4967-9a6c-93d23cecb450'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:23:41.720446 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8f6dac42-b8d0-4a34-84fe-0f9b8e86dee3 req-f8d0a8a2-a072-49db-97b9-90bc920c449d tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:41.721900 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8f6dac42-b8d0-4a34-84fe-0f9b8e86dee3 req-f8d0a8a2-a072-49db-97b9-90bc920c449d tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 931 time: 0.0817735 May 20 04:23:41.767122 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8f6dac42-b8d0-4a34-84fe-0f9b8e86dee3 req-367d9334-8071-4169-826f-4a8204543679 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0389783 May 20 04:23:42.590895 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-b6c97d56-8285-48c3-a926-00e658c216d2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:42.805297 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1375fa47-db7a-498f-82a0-b09cc9a95c71 req-b283cbbf-9221-4d9a-beb8-c8f625abb15e tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:42.807319 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1375fa47-db7a-498f-82a0-b09cc9a95c71 req-b283cbbf-9221-4d9a-beb8-c8f625abb15e tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.1500678 May 20 04:23:42.857517 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1375fa47-db7a-498f-82a0-b09cc9a95c71 req-58a43f6a-0d53-471e-a936-64686d78f6c7 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0428801 May 20 04:23:42.956412 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Request body: {'port': {'device_id': 'bc7b7a9f-a875-49ec-a1ed-47a7798818e7', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:23:43.117251 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:43.168456 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-be4f0261-644d-4421-bd83-bcf73e281a71 req-ea4a67b8-e01b-4710-bba0-55a234373d63 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:43.170130 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-be4f0261-644d-4421-bd83-bcf73e281a71 req-ea4a67b8-e01b-4710-bba0-55a234373d63 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 931 time: 0.1896682 May 20 04:23:43.271634 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-be4f0261-644d-4421-bd83-bcf73e281a71 req-2fa93a5b-2766-434a-abc5-216f01e4fb25 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0954242 May 20 04:23:43.505731 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 643e5696-1a48-46bc-9a75-8b39f176b597 {{(pid=91643) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:23:43.512984 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Notify callbacks [] for port, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:43.527914 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.provisioning_blocks [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Transition to ACTIVE for port object 643e5696-1a48-46bc-9a75-8b39f176b597 will not be triggered until provisioned by entity L2. {{(pid=91643) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:23:43.533945 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:43.537970 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-46425c4d-46b7-4040-a2ba-a7ef270604d1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:43.658955 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Pushing event updated for resources: {'Port': ['ID=643e5696-1a48-46bc-9a75-8b39f176b597,revision_number=2']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:43.684178 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.146s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:43.710309 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Attempting to bind port 643e5696-1a48-46bc-9a75-8b39f176b597 on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:23:43.710634 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Attempting to bind port 643e5696-1a48-46bc-9a75-8b39f176b597 by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': '6fe99325-a72c-48bf-abc7-9bbcc4b753ff', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 836, 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f'}] {{(pid=91643) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:23:43.710961 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Attempting to bind port 643e5696-1a48-46bc-9a75-8b39f176b597 on network 54fc8d38-193b-49c4-b873-c5353723791f {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:23:43.715888 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Port 643e5696-1a48-46bc-9a75-8b39f176b597 on network 54fc8d38-193b-49c4-b873-c5353723791f not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:23:43.716204 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Attempting to bind port 643e5696-1a48-46bc-9a75-8b39f176b597 on network 54fc8d38-193b-49c4-b873-c5353723791f {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:23:43.721981 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 23, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:23:43.722310 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Checking segment: {'id': '6fe99325-a72c-48bf-abc7-9bbcc4b753ff', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 836, 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91643) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:23:43.722679 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Bound using segment: {'id': '6fe99325-a72c-48bf-abc7-9bbcc4b753ff', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 836, 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f'} {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:23:43.723100 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Bound port: 643e5696-1a48-46bc-9a75-8b39f176b597, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '6fe99325-a72c-48bf-abc7-9bbcc4b753ff', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 836, 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f'}}] {{(pid=91643) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:23:43.723461 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:43.821773 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] For port 643e5696-1a48-46bc-9a75-8b39f176b597, host hv-cinder-78240, cleared binding levels {{(pid=91643) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:23:43.824771 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] For port 643e5696-1a48-46bc-9a75-8b39f176b597, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=643e5696-1a48-46bc-9a75-8b39f176b597,segment=NetworkSegment(6fe99325-a72c-48bf-abc7-9bbcc4b753ff),segment_id=6fe99325-a72c-48bf-abc7-9bbcc4b753ff)] {{(pid=91643) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:23:43.830437 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:43.834763 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-44131c8e-ddb4-46c3-b4c4-8cccce7e4eb7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:43.893072 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:43.893999 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:43.896233 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] 10.222.0.74 "PUT /v2.0/ports/643e5696-1a48-46bc-9a75-8b39f176b597 HTTP/1.1" status: 200 len: 1093 time: 0.9444101 May 20 04:23:43.896705 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'bc7b7a9f-a875-49ec-a1ed-47a7798818e7', 'tag': '643e5696-1a48-46bc-9a75-8b39f176b597'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:23:43.915474 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Pushing event updated for resources: {'Port': ['ID=643e5696-1a48-46bc-9a75-8b39f176b597,revision_number=3']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:43.918723 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:44.049184 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c457edee-82da-47fe-963a-c2e00ca6db49 req-acdffe08-49e0-45a8-a60c-7ca7ddb8685f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:44.051153 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c457edee-82da-47fe-963a-c2e00ca6db49 req-acdffe08-49e0-45a8-a60c-7ca7ddb8685f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0859618 May 20 04:23:44.097266 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c457edee-82da-47fe-963a-c2e00ca6db49 req-d62c5bfe-577b-4595-8f55-a6fe5bc1e2d9 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0401268 May 20 04:23:44.147044 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'bc7b7a9f-a875-49ec-a1ed-47a7798818e7', 'tag': '643e5696-1a48-46bc-9a75-8b39f176b597', 'status': 'completed', 'code': 200} May 20 04:23:44.478010 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2b1f1af8-bcd1-429f-949d-e3ded05bc196 req-cb24f0b8-3a9a-4211-b9e1-16373c6a1eea tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:44.479388 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b1f1af8-bcd1-429f-949d-e3ded05bc196 req-cb24f0b8-3a9a-4211-b9e1-16373c6a1eea tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 943 time: 0.0897431 May 20 04:23:44.529346 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b1f1af8-bcd1-429f-949d-e3ded05bc196 req-10e9b802-4260-4c1e-b885-e14c21e276cf tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0414979 May 20 04:23:44.953816 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:45.017633 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2450ac37-d256-4c10-bc4c-4e464de4a873 req-af47c7a2-f91a-41dd-be1e-97245f3e2d5c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:45.019133 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2450ac37-d256-4c10-bc4c-4e464de4a873 req-af47c7a2-f91a-41dd-be1e-97245f3e2d5c service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=51fcbc78284541db832e4780e76158af&device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 1096 time: 0.0756814 May 20 04:23:45.072361 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-7ae2584d-e722-4c9d-8e42-8ccfc7a6995a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:45.166078 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2450ac37-d256-4c10-bc4c-4e464de4a873 req-e6e67bc9-094a-40a2-bf5e-7690666a37da service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:45.167547 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2450ac37-d256-4c10-bc4c-4e464de4a873 req-e6e67bc9-094a-40a2-bf5e-7690666a37da service neutron] 10.222.0.74 "GET /v2.0/networks?tenant_id=51fcbc78284541db832e4780e76158af&shared=False HTTP/1.1" status: 200 len: 916 time: 0.1373973 May 20 04:23:45.355612 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2450ac37-d256-4c10-bc4c-4e464de4a873 req-fa66e6cc-90d0-40da-a064-116fbe81e359 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:45.357035 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2450ac37-d256-4c10-bc4c-4e464de4a873 req-fa66e6cc-90d0-40da-a064-116fbe81e359 service neutron] 10.222.0.74 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.1826782 May 20 04:23:45.372478 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7f11afd3-3846-44b1-891e-2868b003da14 req-fdc492cc-cf65-41a8-a664-1066430d4ec0 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:45.373853 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7f11afd3-3846-44b1-891e-2868b003da14 req-fdc492cc-cf65-41a8-a664-1066430d4ec0 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.1667714 May 20 04:23:45.520237 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7f11afd3-3846-44b1-891e-2868b003da14 req-20fe9f15-859b-4e08-b2b4-6509311c8b85 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1385207 May 20 04:23:45.540550 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-b89c95d7-311d-4f4e-b4de-65a89b58e544 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:45.541996 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-b89c95d7-311d-4f4e-b4de-65a89b58e544 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=51fcbc78284541db832e4780e76158af&device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 1096 time: 0.0901504 May 20 04:23:45.739482 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e8a3009f-1b79-4a19-8dc4-6facc58ebac4 req-4ba8be28-2f79-47d0-882a-cc7d5aa8ed82 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:45.740955 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e8a3009f-1b79-4a19-8dc4-6facc58ebac4 req-4ba8be28-2f79-47d0-882a-cc7d5aa8ed82 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 943 time: 0.0870626 May 20 04:23:45.838015 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-576a896a-9516-4ff5-8511-a173b191c04a service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=643e5696-1a48-46bc-9a75-8b39f176b597 HTTP/1.1" status: 200 len: 217 time: 0.0776312 May 20 04:23:45.887501 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e8a3009f-1b79-4a19-8dc4-6facc58ebac4 req-b64adc7a-9a4c-42fe-996b-701a00ae489e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1405771 May 20 04:23:45.908520 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-a43ffd9c-ac62-41ef-839d-3ae256a4d1a4 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:45.910019 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-a43ffd9c-ac62-41ef-839d-3ae256a4d1a4 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=0f918085-8d33-4e36-a59e-7f270a2b1874 HTTP/1.1" status: 200 len: 855 time: 0.0652361 May 20 04:23:45.968535 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-b97c5a0f-c8ea-4bfc-88fe-f7d1723bf998 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=54fc8d38-193b-49c4-b873-c5353723791f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0509059 May 20 04:23:46.103920 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-88cd0186-4a2c-495c-83d5-6197f01e1e9e service neutron] 10.222.0.74 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1262813 May 20 04:23:46.174330 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:46.237022 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3edc9e95-1795-47dd-831d-43356d80a5b4 req-d09d3d10-235a-4f2f-a023-df107a416975 service neutron] 10.222.0.74 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1264982 May 20 04:23:46.713463 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-09925ace-2783-4f31-b73b-2396ed30242f req-cd9dfc11-19ad-4f09-bf35-50bc6b82aedf tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:46.715025 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-09925ace-2783-4f31-b73b-2396ed30242f req-cd9dfc11-19ad-4f09-bf35-50bc6b82aedf tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0829306 May 20 04:23:46.759657 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-09925ace-2783-4f31-b73b-2396ed30242f req-5005a41d-4360-46c0-b835-5b8bd3a3923c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0384696 May 20 04:23:47.101472 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5526c842-b12c-4976-b5cf-9836f829c120 req-26dac551-3e02-42cc-b83f-364fa1c2c4b1 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:47.102978 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5526c842-b12c-4976-b5cf-9836f829c120 req-26dac551-3e02-42cc-b83f-364fa1c2c4b1 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 943 time: 0.0849168 May 20 04:23:47.156881 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5526c842-b12c-4976-b5cf-9836f829c120 req-d50c1b41-6777-47bf-a0b2-ffccf38ae3c6 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0471961 May 20 04:23:47.419609 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 35730) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:47.550466 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5657f594-f480-4bd7-b0d6-49b82791f43f admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:47.567147 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-5657f594-f480-4bd7-b0d6-49b82791f43f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:23:47.571355 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-5657f594-f480-4bd7-b0d6-49b82791f43f admin admin] Usage tracker for resource:security_group_rule and tenant:e0d6fdb98a3449a78c84c0f4dee1af02 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:23:47.574153 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-5657f594-f480-4bd7-b0d6-49b82791f43f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:23:47.579961 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-5657f594-f480-4bd7-b0d6-49b82791f43f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:23:47.662469 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5657f594-f480-4bd7-b0d6-49b82791f43f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:47.666797 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5657f594-f480-4bd7-b0d6-49b82791f43f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:47.675182 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-ebce0eea-a0e7-4c14-bfa7-fe9b9900d07f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:47.714430 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-5657f594-f480-4bd7-b0d6-49b82791f43f admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=e0d6fdb98a3449a78c84c0f4dee1af02&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2936163 May 20 04:23:47.715358 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5657f594-f480-4bd7-b0d6-49b82791f43f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=6e124b73-4d5c-423c-8ac6-8613e12192d0,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:47.717368 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 35748) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:47.718892 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-5657f594-f480-4bd7-b0d6-49b82791f43f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:47.783482 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8d139106-40c5-4d89-a0d0-3d455676bd4b admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:47.797467 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:47.814917 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8d139106-40c5-4d89-a0d0-3d455676bd4b admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:47.874574 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8d139106-40c5-4d89-a0d0-3d455676bd4b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:47.881276 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:47.894486 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8d139106-40c5-4d89-a0d0-3d455676bd4b admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=6e124b73-4d5c-423c-8ac6-8613e12192d0,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:47.899459 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-8d139106-40c5-4d89-a0d0-3d455676bd4b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:47.903550 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8d139106-40c5-4d89-a0d0-3d455676bd4b admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:47.907101 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-8d139106-40c5-4d89-a0d0-3d455676bd4b admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/6e124b73-4d5c-423c-8ac6-8613e12192d0 HTTP/1.1" status: 204 len: 168 time: 0.1853721 May 20 04:23:47.977477 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ccd0ae3b-c4a5-422e-8da5-b0bdf0365064 req-8986d906-eded-4a70-821b-c9a081a11d44 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:47.979239 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ccd0ae3b-c4a5-422e-8da5-b0bdf0365064 req-8986d906-eded-4a70-821b-c9a081a11d44 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.1017711 May 20 04:23:48.024987 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ccd0ae3b-c4a5-422e-8da5-b0bdf0365064 req-6d1e711e-3bc6-4b76-92f7-b7283370f3ac tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0396621 May 20 04:23:48.210363 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 35766) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:48.228552 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-95b20e09-1bd8-4148-bf2f-abc4d2e23613 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:48.243435 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-95b20e09-1bd8-4148-bf2f-abc4d2e23613 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:23:48.247715 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-95b20e09-1bd8-4148-bf2f-abc4d2e23613 admin admin] Usage tracker for resource:security_group_rule and tenant:5c8d2a1b67d448249fff618baf9f80d2 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:23:48.250570 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-95b20e09-1bd8-4148-bf2f-abc4d2e23613 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:23:48.259276 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-95b20e09-1bd8-4148-bf2f-abc4d2e23613 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:23:48.348359 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-95b20e09-1bd8-4148-bf2f-abc4d2e23613 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:48.353677 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-95b20e09-1bd8-4148-bf2f-abc4d2e23613 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:48.360667 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1b787960-0954-4e1b-91ed-a5566e04591c req-d15b3a3c-6c57-4506-b295-e26f325f7cd9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:48.362102 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1b787960-0954-4e1b-91ed-a5566e04591c req-d15b3a3c-6c57-4506-b295-e26f325f7cd9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 943 time: 0.0910392 May 20 04:23:48.366738 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-2a05e45e-0628-4cf3-8431-3e5337def7de admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:48.421628 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1b787960-0954-4e1b-91ed-a5566e04591c req-0cccc12f-d30b-468d-a837-1f62d9bf9404 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0531781 May 20 04:23:48.424473 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 35772) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:48.424620 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-95b20e09-1bd8-4148-bf2f-abc4d2e23613 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=5c8d2a1b67d448249fff618baf9f80d2&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2122338 May 20 04:23:48.426462 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-95b20e09-1bd8-4148-bf2f-abc4d2e23613 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=50658b3c-4970-429c-95df-864366f6d3dd,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:48.429921 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-95b20e09-1bd8-4148-bf2f-abc4d2e23613 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.063s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:48.491897 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-738b0e7d-f542-4d42-a53c-eef12f5aca28 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:48.518467 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-738b0e7d-f542-4d42-a53c-eef12f5aca28 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:48.572692 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-738b0e7d-f542-4d42-a53c-eef12f5aca28 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:48.576241 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-7c6577f3-c9b1-4da8-8aa9-a53681d9cad4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:48.588985 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-738b0e7d-f542-4d42-a53c-eef12f5aca28 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=50658b3c-4970-429c-95df-864366f6d3dd,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:48.593688 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-738b0e7d-f542-4d42-a53c-eef12f5aca28 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:48.598083 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-738b0e7d-f542-4d42-a53c-eef12f5aca28 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:48.599550 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-738b0e7d-f542-4d42-a53c-eef12f5aca28 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/50658b3c-4970-429c-95df-864366f6d3dd HTTP/1.1" status: 204 len: 168 time: 0.1743956 May 20 04:23:49.242260 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0c0b9b0b-2457-4ea2-a49e-e7d13d037222 req-c13e2ab9-bcd0-4dcc-a6fb-d71b9900fe9e tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:49.243846 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0c0b9b0b-2457-4ea2-a49e-e7d13d037222 req-c13e2ab9-bcd0-4dcc-a6fb-d71b9900fe9e tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.1002219 May 20 04:23:49.270970 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 35814) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:49.301101 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0c0b9b0b-2457-4ea2-a49e-e7d13d037222 req-2ebc8826-d50a-451a-b216-5586524b022b tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0491889 May 20 04:23:49.398495 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1ee75548-3032-42bd-b854-6b7da8d85f35 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:49.413190 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-1ee75548-3032-42bd-b854-6b7da8d85f35 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:23:49.417165 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-1ee75548-3032-42bd-b854-6b7da8d85f35 admin admin] Usage tracker for resource:security_group_rule and tenant:c1f83f418da0409188b2cd16dfc5c156 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:23:49.420011 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-1ee75548-3032-42bd-b854-6b7da8d85f35 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:23:49.425733 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-1ee75548-3032-42bd-b854-6b7da8d85f35 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:23:49.554034 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1ee75548-3032-42bd-b854-6b7da8d85f35 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:49.558400 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1ee75548-3032-42bd-b854-6b7da8d85f35 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:49.584408 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d5777953-7f9d-41d0-b8ec-d6c7583cde32 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:49.608241 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-1ee75548-3032-42bd-b854-6b7da8d85f35 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=c1f83f418da0409188b2cd16dfc5c156&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3349016 May 20 04:23:49.610690 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 35834) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:49.613326 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1ee75548-3032-42bd-b854-6b7da8d85f35 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=0f846b9f-4d9d-4897-a228-ad57807a57ce,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:49.616189 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-1ee75548-3032-42bd-b854-6b7da8d85f35 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.032s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:49.707259 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8fb459cd-2b0f-49ac-b6f1-f17ac796e065 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:49.715559 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-948f580e-0352-4d51-95b1-e7a04db65db1 req-586c7244-1ef2-4024-9102-7070dd55c3dc tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:49.717247 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-948f580e-0352-4d51-95b1-e7a04db65db1 req-586c7244-1ef2-4024-9102-7070dd55c3dc tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 943 time: 0.1711602 May 20 04:23:49.760682 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8fb459cd-2b0f-49ac-b6f1-f17ac796e065 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:49.796363 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-948f580e-0352-4d51-95b1-e7a04db65db1 req-d9e06995-fbb8-4ae6-ae87-4f6b1b627e5e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0707388 May 20 04:23:49.799118 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 643e5696-1a48-46bc-9a75-8b39f176b597 details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:23:49.833864 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8fb459cd-2b0f-49ac-b6f1-f17ac796e065 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:49.840710 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-227f2247-4500-4943-936d-b55e94501b75 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:49.855046 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8fb459cd-2b0f-49ac-b6f1-f17ac796e065 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=0f846b9f-4d9d-4897-a228-ad57807a57ce,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:49.860248 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-8fb459cd-2b0f-49ac-b6f1-f17ac796e065 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:49.861242 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8fb459cd-2b0f-49ac-b6f1-f17ac796e065 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:49.862282 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-8fb459cd-2b0f-49ac-b6f1-f17ac796e065 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/0f846b9f-4d9d-4897-a228-ad57807a57ce HTTP/1.1" status: 204 len: 168 time: 0.2485583 May 20 04:23:50.007909 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 643e5696-1a48-46bc-9a75-8b39f176b597, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=643e5696-1a48-46bc-9a75-8b39f176b597,segment=NetworkSegment(6fe99325-a72c-48bf-abc7-9bbcc4b753ff),segment_id=6fe99325-a72c-48bf-abc7-9bbcc4b753ff)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:23:50.024796 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': '643e5696-1a48-46bc-9a75-8b39f176b597', 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f', 'port_id': '643e5696-1a48-46bc-9a75-8b39f176b597', 'mac_address': 'fa:16:3e:f7:27:ef', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 836, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '0f918085-8d33-4e36-a59e-7f270a2b1874', 'ip_address': '172.20.0.3'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:23:50.088909 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:50.110535 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 643e5696-1a48-46bc-9a75-8b39f176b597 {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:23:50.127378 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 643e5696-1a48-46bc-9a75-8b39f176b597, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=643e5696-1a48-46bc-9a75-8b39f176b597,segment=NetworkSegment(6fe99325-a72c-48bf-abc7-9bbcc4b753ff),segment_id=6fe99325-a72c-48bf-abc7-9bbcc4b753ff)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:23:50.133082 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:50.135851 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:50.206419 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=643e5696-1a48-46bc-9a75-8b39f176b597,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:50.214432 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:50.214882 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:50.215497 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:50.215698 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:50.216281 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:50.217207 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:50.217678 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:50.218160 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:50.218641 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:50.219111 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:50.219608 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:50.220067 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:50.225624 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.090s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:50.492763 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fb6c2418-3043-423f-aa63-3d51a1dd6c74 req-3cadc28a-ea39-4750-afc0-ba404be8c3dd tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:50.494278 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fb6c2418-3043-423f-aa63-3d51a1dd6c74 req-3cadc28a-ea39-4750-afc0-ba404be8c3dd tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0802310 May 20 04:23:50.606678 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 35890) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:50.640067 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 35896) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:50.658742 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fb6c2418-3043-423f-aa63-3d51a1dd6c74 req-1e5be5e2-5cb2-4465-a66e-6227c75f1a86 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1544189 May 20 04:23:50.766334 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['643e5696-1a48-46bc-9a75-8b39f176b597'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:23:50.821207 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:50.837695 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:23:50.841005 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-27514652-fb14-4cb3-9f26-5c7459ad77cc admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:50.872850 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Usage tracker for resource:security_group_rule and tenant:0dd4cd72c42841179a31a65f1f49f7c9 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:23:50.879569 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:23:50.891793 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:23:50.894215 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-27514652-fb14-4cb3-9f26-5c7459ad77cc admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:23:50.901080 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-27514652-fb14-4cb3-9f26-5c7459ad77cc admin admin] Usage tracker for resource:security_group_rule and tenant:0dd416bb813249509c9b0d82c4e46bf4 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:23:50.905328 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-27514652-fb14-4cb3-9f26-5c7459ad77cc admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:23:50.916205 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-27514652-fb14-4cb3-9f26-5c7459ad77cc admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:23:51.179050 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:51.189650 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:51.200794 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-0c7c941d-f2cc-4283-8527-820027d5a52c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:51.249740 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-27514652-fb14-4cb3-9f26-5c7459ad77cc admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:51.257265 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6c85e3fe-a6c4-4b33-920f-8633bc8d5ae3 req-7008a004-d22c-4f25-9cdd-80b009fa4bba tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:51.258937 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6c85e3fe-a6c4-4b33-920f-8633bc8d5ae3 req-7008a004-d22c-4f25-9cdd-80b009fa4bba tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 944 time: 0.0980372 May 20 04:23:51.264157 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-27514652-fb14-4cb3-9f26-5c7459ad77cc admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:51.277528 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=9dca5c62-2309-4f59-b06d-44b004745b07,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:51.285212 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:51.285577 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:51.285927 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:51.286766 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:51.287130 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:51.287505 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:51.287846 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:51.288332 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:51.288679 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:51.289018 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:51.289345 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:51.289673 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:51.293712 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.093s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:51.298793 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=0dd4cd72c42841179a31a65f1f49f7c9&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6887009 May 20 04:23:51.300016 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.027s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:51.301459 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 35932) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:51.323255 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6c85e3fe-a6c4-4b33-920f-8633bc8d5ae3 req-bb5c8d71-cd3f-4469-ba0e-d2293211acb8 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0561843 May 20 04:23:51.339967 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-27514652-fb14-4cb3-9f26-5c7459ad77cc admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=0dd416bb813249509c9b0d82c4e46bf4&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6983175 May 20 04:23:51.352629 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 35936) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:51.353317 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-27514652-fb14-4cb3-9f26-5c7459ad77cc admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=0fa877b9-f58a-4d64-af69-6641c2ccb728,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:51.357575 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-27514652-fb14-4cb3-9f26-5c7459ad77cc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.057s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:51.386021 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f25e1a99-0fce-4348-85d4-9b1f205ecb45 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:51.412260 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f25e1a99-0fce-4348-85d4-9b1f205ecb45 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:51.433415 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-777f2221-5211-43b3-90d7-177d0c2c3203 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:51.462818 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-777f2221-5211-43b3-90d7-177d0c2c3203 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:51.470482 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f25e1a99-0fce-4348-85d4-9b1f205ecb45 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:51.476799 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:51.490174 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f25e1a99-0fce-4348-85d4-9b1f205ecb45 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=9dca5c62-2309-4f59-b06d-44b004745b07,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:51.493433 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-f25e1a99-0fce-4348-85d4-9b1f205ecb45 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:51.496769 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f25e1a99-0fce-4348-85d4-9b1f205ecb45 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:51.498120 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-f25e1a99-0fce-4348-85d4-9b1f205ecb45 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/9dca5c62-2309-4f59-b06d-44b004745b07 HTTP/1.1" status: 204 len: 168 time: 0.1960273 May 20 04:23:51.536772 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-777f2221-5211-43b3-90d7-177d0c2c3203 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:51.550531 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-2643d934-63c1-44c7-84ec-c9d96327bdd3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:51.567035 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-777f2221-5211-43b3-90d7-177d0c2c3203 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=0fa877b9-f58a-4d64-af69-6641c2ccb728,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:51.570041 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-777f2221-5211-43b3-90d7-177d0c2c3203 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.022s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:51.583165 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-777f2221-5211-43b3-90d7-177d0c2c3203 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:51.584580 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-777f2221-5211-43b3-90d7-177d0c2c3203 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/0fa877b9-f58a-4d64-af69-6641c2ccb728 HTTP/1.1" status: 204 len: 168 time: 0.2270172 May 20 04:23:51.972142 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a08a3419-5462-400a-a1e4-cc85dbdcd0bd req-49df02d2-e6bd-47cc-8419-2f88a6c89909 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:51.975267 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a08a3419-5462-400a-a1e4-cc85dbdcd0bd req-49df02d2-e6bd-47cc-8419-2f88a6c89909 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.1559544 May 20 04:23:52.057010 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a08a3419-5462-400a-a1e4-cc85dbdcd0bd req-40be7980-30a0-4dc4-9ec8-a888822ae33f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0718541 May 20 04:23:52.153389 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 35990) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:52.400137 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c4486dbf-3e24-455a-aebe-939887fe0784 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:52.415838 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-c4486dbf-3e24-455a-aebe-939887fe0784 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:23:52.420498 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-c4486dbf-3e24-455a-aebe-939887fe0784 admin admin] Usage tracker for resource:security_group_rule and tenant:c74d308331e246488e366aaead71245a is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:23:52.423855 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-c4486dbf-3e24-455a-aebe-939887fe0784 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:23:52.435550 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-c4486dbf-3e24-455a-aebe-939887fe0784 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:23:52.542511 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c4486dbf-3e24-455a-aebe-939887fe0784 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:52.549787 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c4486dbf-3e24-455a-aebe-939887fe0784 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:52.567658 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:52.572106 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fae45e19-675d-4e33-9fb2-e60c3b366b23 req-7a322135-f18d-4563-9cfe-ebbf2b4c7cd1 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:52.573499 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fae45e19-675d-4e33-9fb2-e60c3b366b23 req-7a322135-f18d-4563-9cfe-ebbf2b4c7cd1 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 944 time: 0.1014748 May 20 04:23:52.613650 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 36020) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:52.614659 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-c4486dbf-3e24-455a-aebe-939887fe0784 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=c74d308331e246488e366aaead71245a&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4588194 May 20 04:23:52.619775 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c4486dbf-3e24-455a-aebe-939887fe0784 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=5e855af3-61c6-4d0f-97d0-cb1abbb5a0b7,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:52.622864 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-c4486dbf-3e24-455a-aebe-939887fe0784 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:52.637364 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fae45e19-675d-4e33-9fb2-e60c3b366b23 req-780deb5e-936c-4d4c-99da-7df6cddc7bd5 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0567603 May 20 04:23:52.685131 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8cbff7f5-95bc-4c6d-9de1-ee6f26366cf7 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:52.711434 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8cbff7f5-95bc-4c6d-9de1-ee6f26366cf7 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:52.766076 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8cbff7f5-95bc-4c6d-9de1-ee6f26366cf7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:52.771691 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-f25e1a99-0fce-4348-85d4-9b1f205ecb45 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:52.783462 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8cbff7f5-95bc-4c6d-9de1-ee6f26366cf7 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=5e855af3-61c6-4d0f-97d0-cb1abbb5a0b7,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:52.793102 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8cbff7f5-95bc-4c6d-9de1-ee6f26366cf7 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:52.793102 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8cbff7f5-95bc-4c6d-9de1-ee6f26366cf7 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:52.793412 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8cbff7f5-95bc-4c6d-9de1-ee6f26366cf7 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:52.793740 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8cbff7f5-95bc-4c6d-9de1-ee6f26366cf7 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:52.793960 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8cbff7f5-95bc-4c6d-9de1-ee6f26366cf7 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:52.794297 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8cbff7f5-95bc-4c6d-9de1-ee6f26366cf7 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:52.794806 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8cbff7f5-95bc-4c6d-9de1-ee6f26366cf7 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:52.795111 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8cbff7f5-95bc-4c6d-9de1-ee6f26366cf7 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:52.795463 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8cbff7f5-95bc-4c6d-9de1-ee6f26366cf7 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:52.795855 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8cbff7f5-95bc-4c6d-9de1-ee6f26366cf7 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:52.796191 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8cbff7f5-95bc-4c6d-9de1-ee6f26366cf7 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:52.796517 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8cbff7f5-95bc-4c6d-9de1-ee6f26366cf7 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:23:52.799707 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8cbff7f5-95bc-4c6d-9de1-ee6f26366cf7 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:52.801109 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-8cbff7f5-95bc-4c6d-9de1-ee6f26366cf7 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/5e855af3-61c6-4d0f-97d0-cb1abbb5a0b7 HTTP/1.1" status: 204 len: 168 time: 0.1865079 May 20 04:23:52.802426 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-8cbff7f5-95bc-4c6d-9de1-ee6f26366cf7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.031s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:53.240942 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8a41ef63-223f-4609-bfee-68ac7b318f4b req-0e059057-ce83-4978-ade5-bf99ebf01948 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:53.242412 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8a41ef63-223f-4609-bfee-68ac7b318f4b req-0e059057-ce83-4978-ade5-bf99ebf01948 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0832653 May 20 04:23:53.296626 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8a41ef63-223f-4609-bfee-68ac7b318f4b req-1ec9968d-77a7-455c-85cb-516deca52216 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0485697 May 20 04:23:53.340985 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 36058) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:53.360098 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-58148a96-e9d6-494e-b4f4-c61576684f71 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:53.376358 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-58148a96-e9d6-494e-b4f4-c61576684f71 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:23:53.381052 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-58148a96-e9d6-494e-b4f4-c61576684f71 admin admin] Usage tracker for resource:security_group_rule and tenant:1ed6d2a93efc43e3b8de4396956aad3a is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:23:53.384175 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-58148a96-e9d6-494e-b4f4-c61576684f71 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:23:53.390730 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-58148a96-e9d6-494e-b4f4-c61576684f71 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:23:53.491180 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-58148a96-e9d6-494e-b4f4-c61576684f71 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:53.495759 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-58148a96-e9d6-494e-b4f4-c61576684f71 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:53.502750 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:53.546578 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-58148a96-e9d6-494e-b4f4-c61576684f71 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e035c4d0-b318-486e-b416-d8178fa9679f,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:53.564450 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-58148a96-e9d6-494e-b4f4-c61576684f71 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.062s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:53.576102 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-58148a96-e9d6-494e-b4f4-c61576684f71 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=1ed6d2a93efc43e3b8de4396956aad3a&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2339849 May 20 04:23:53.578691 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 36072) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:53.660812 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-99a62b74-28b7-4461-9ff4-c9328c07b4c2 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:53.696733 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-99a62b74-28b7-4461-9ff4-c9328c07b4c2 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:53.771634 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-99a62b74-28b7-4461-9ff4-c9328c07b4c2 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:53.783096 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:53.800948 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:23:53.826132 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-99a62b74-28b7-4461-9ff4-c9328c07b4c2 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:53.831581 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-99a62b74-28b7-4461-9ff4-c9328c07b4c2 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/e035c4d0-b318-486e-b416-d8178fa9679f HTTP/1.1" status: 204 len: 168 time: 0.2498200 May 20 04:23:53.834258 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-99a62b74-28b7-4461-9ff4-c9328c07b4c2 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e035c4d0-b318-486e-b416-d8178fa9679f,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:53.834977 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 643e5696-1a48-46bc-9a75-8b39f176b597 up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:23:53.839068 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-99a62b74-28b7-4461-9ff4-c9328c07b4c2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.058s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:53.930750 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port 643e5696-1a48-46bc-9a75-8b39f176b597 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:23:53.942908 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port 643e5696-1a48-46bc-9a75-8b39f176b597 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:23:53.943323 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:53.945974 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d7f6ce95-b746-4fd2-9d99-39f66709f975 req-06bbfa6e-45fe-451c-896e-6da0bf2f17bf tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:53.947720 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d7f6ce95-b746-4fd2-9d99-39f66709f975 req-06bbfa6e-45fe-451c-896e-6da0bf2f17bf tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 944 time: 0.1643088 May 20 04:23:54.002185 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d7f6ce95-b746-4fd2-9d99-39f66709f975 req-ae8635e8-b926-4e2a-b5f2-a0c00cbce1f8 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0476463 May 20 04:23:54.058931 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 36086) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:54.068037 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:54.112884 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'bc7b7a9f-a875-49ec-a1ed-47a7798818e7', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '643e5696-1a48-46bc-9a75-8b39f176b597'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:23:54.139325 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 643e5696-1a48-46bc-9a75-8b39f176b597, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=643e5696-1a48-46bc-9a75-8b39f176b597,segment=NetworkSegment(6fe99325-a72c-48bf-abc7-9bbcc4b753ff),segment_id=6fe99325-a72c-48bf-abc7-9bbcc4b753ff)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:23:54.146185 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:54.152285 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:54.211734 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'bc7b7a9f-a875-49ec-a1ed-47a7798818e7', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '643e5696-1a48-46bc-9a75-8b39f176b597', 'code': 200} May 20 04:23:54.217998 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-707cc159-8df8-4bc9-9eef-2f124c5a7a97 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:54.238140 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-707cc159-8df8-4bc9-9eef-2f124c5a7a97 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:23:54.242326 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-707cc159-8df8-4bc9-9eef-2f124c5a7a97 admin admin] Usage tracker for resource:security_group_rule and tenant:60aed755450842b68ecee63bf0c6fc69 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:23:54.245448 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-707cc159-8df8-4bc9-9eef-2f124c5a7a97 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:23:54.250923 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=643e5696-1a48-46bc-9a75-8b39f176b597,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:54.253742 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-707cc159-8df8-4bc9-9eef-2f124c5a7a97 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:23:54.254251 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:54.364343 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-707cc159-8df8-4bc9-9eef-2f124c5a7a97 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:54.368634 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-707cc159-8df8-4bc9-9eef-2f124c5a7a97 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:54.378297 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:54.424198 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-707cc159-8df8-4bc9-9eef-2f124c5a7a97 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=cff5e5e9-631f-48a5-bf59-3cc78411a314,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:54.434298 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-707cc159-8df8-4bc9-9eef-2f124c5a7a97 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.056s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:54.439711 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-707cc159-8df8-4bc9-9eef-2f124c5a7a97 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=60aed755450842b68ecee63bf0c6fc69&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3779917 May 20 04:23:54.440945 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 36110) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:54.507971 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-5a3732e2-50c2-4016-8672-c94f27a8b7cc admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:54.533963 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-5a3732e2-50c2-4016-8672-c94f27a8b7cc admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:54.553820 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5c37a15f-5930-4880-9cec-f578cea7d8c2 req-630a9af9-345c-470d-9a33-d95489075b84 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:54.555272 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5c37a15f-5930-4880-9cec-f578cea7d8c2 req-630a9af9-345c-470d-9a33-d95489075b84 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 1098 time: 0.1543236 May 20 04:23:54.567087 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fff58bcd-b306-4dd2-9d15-0aea5d3429ba req-ce1bcb2b-9f1a-4e5e-bfaf-8be347112b58 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:54.568819 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fff58bcd-b306-4dd2-9d15-0aea5d3429ba req-ce1bcb2b-9f1a-4e5e-bfaf-8be347112b58 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.1104841 May 20 04:23:54.599497 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-5a3732e2-50c2-4016-8672-c94f27a8b7cc admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:54.605110 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-777f2221-5211-43b3-90d7-177d0c2c3203 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:54.615900 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5a3732e2-50c2-4016-8672-c94f27a8b7cc admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=cff5e5e9-631f-48a5-bf59-3cc78411a314,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:54.621648 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-5a3732e2-50c2-4016-8672-c94f27a8b7cc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:54.624110 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-5a3732e2-50c2-4016-8672-c94f27a8b7cc admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:54.628150 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-5a3732e2-50c2-4016-8672-c94f27a8b7cc admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/cff5e5e9-631f-48a5-bf59-3cc78411a314 HTTP/1.1" status: 204 len: 168 time: 0.1861162 May 20 04:23:54.663485 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5c37a15f-5930-4880-9cec-f578cea7d8c2 req-40cd147c-831e-40a6-b3b2-065ff9902498 service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0812991 May 20 04:23:54.671741 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fff58bcd-b306-4dd2-9d15-0aea5d3429ba req-1b5bfe39-c118-4e68-b956-2afeb43e7b07 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0689952 May 20 04:23:55.219155 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9e8f8f90-31e4-45d9-b579-755c6f5c8ca9 req-2ba85877-ea0b-434b-ac98-7124a3b8e6a5 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:55.221274 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9e8f8f90-31e4-45d9-b579-755c6f5c8ca9 req-2ba85877-ea0b-434b-ac98-7124a3b8e6a5 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.1907485 May 20 04:23:55.287346 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9e8f8f90-31e4-45d9-b579-755c6f5c8ca9 req-851175de-0e39-4212-b10d-110c82de30ce tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0570323 May 20 04:23:55.310151 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-78a24143-1a97-47d1-a2a4-ab9714a28e18 req-8ea441f3-1fed-4a17-b5c5-860abd866a32 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:55.312769 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-78a24143-1a97-47d1-a2a4-ab9714a28e18 req-8ea441f3-1fed-4a17-b5c5-860abd866a32 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 945 time: 0.1660278 May 20 04:23:55.391551 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-78a24143-1a97-47d1-a2a4-ab9714a28e18 req-b8a7c171-eb9c-40d5-98fd-032f8447a1a6 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0712063 May 20 04:23:55.875784 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d6b8786c-eb84-4bd3-8564-2606619b011e req-9ccf4a17-ed2e-4652-900a-ab9e27b41509 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:55.877333 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d6b8786c-eb84-4bd3-8564-2606619b011e req-9ccf4a17-ed2e-4652-900a-ab9e27b41509 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0862966 May 20 04:23:55.928674 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d6b8786c-eb84-4bd3-8564-2606619b011e req-5c008f25-f771-4e40-9240-e1d1b8821468 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0441427 May 20 04:23:56.500375 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7ed1017c-c6e1-401e-b3f1-8592436a7a1e req-9fb89dae-42ce-48c4-aa6a-2c0683fd5d5a tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:56.501813 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7ed1017c-c6e1-401e-b3f1-8592436a7a1e req-9fb89dae-42ce-48c4-aa6a-2c0683fd5d5a tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.0842257 May 20 04:23:56.546411 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7ed1017c-c6e1-401e-b3f1-8592436a7a1e req-230a482e-bb40-4f6f-b7a4-0cbb7b2ed993 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0385523 May 20 04:23:56.565513 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 36260) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:56.684262 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-917cdfa3-6f49-4708-aa5d-da00916d70e8 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:56.698405 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-917cdfa3-6f49-4708-aa5d-da00916d70e8 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:23:56.702187 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-917cdfa3-6f49-4708-aa5d-da00916d70e8 admin admin] Usage tracker for resource:security_group_rule and tenant:524b2fcd3c774802a668c96b5f730e7e is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:23:56.704967 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-917cdfa3-6f49-4708-aa5d-da00916d70e8 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:23:56.710631 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-917cdfa3-6f49-4708-aa5d-da00916d70e8 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:23:56.718060 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f5cf83b7-f20c-49c5-beb4-be479c9c8f05 req-42f2598d-788a-404b-921f-53ddd9a83b4d tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:56.719484 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f5cf83b7-f20c-49c5-beb4-be479c9c8f05 req-42f2598d-788a-404b-921f-53ddd9a83b4d tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 945 time: 0.0786846 May 20 04:23:56.768797 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f5cf83b7-f20c-49c5-beb4-be479c9c8f05 req-123e7563-5f72-4834-9f36-b1143c02d785 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0433946 May 20 04:23:56.793123 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-917cdfa3-6f49-4708-aa5d-da00916d70e8 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:56.798204 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-917cdfa3-6f49-4708-aa5d-da00916d70e8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:56.810045 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-1ee75548-3032-42bd-b854-6b7da8d85f35 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:56.847629 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-917cdfa3-6f49-4708-aa5d-da00916d70e8 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=524b2fcd3c774802a668c96b5f730e7e&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2811973 May 20 04:23:56.849213 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-917cdfa3-6f49-4708-aa5d-da00916d70e8 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=40dcd50d-9f4b-4d68-aa4c-463920600d07,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:56.850587 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 36266) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:56.852638 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-917cdfa3-6f49-4708-aa5d-da00916d70e8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:56.911367 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-465f8d8e-ef89-4d7a-b889-543212f92987 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:56.941897 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-465f8d8e-ef89-4d7a-b889-543212f92987 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:56.996080 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-465f8d8e-ef89-4d7a-b889-543212f92987 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:57.000804 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:57.012701 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-465f8d8e-ef89-4d7a-b889-543212f92987 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=40dcd50d-9f4b-4d68-aa4c-463920600d07,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:57.016560 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-465f8d8e-ef89-4d7a-b889-543212f92987 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:57.022951 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-465f8d8e-ef89-4d7a-b889-543212f92987 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:57.027207 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-465f8d8e-ef89-4d7a-b889-543212f92987 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/40dcd50d-9f4b-4d68-aa4c-463920600d07 HTTP/1.1" status: 204 len: 168 time: 0.1750252 May 20 04:23:57.128172 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6139ae73-12e3-4f2d-ad01-df32ad96807b req-79b3af43-f9a8-40fc-bcde-5934b89457b9 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:57.129636 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6139ae73-12e3-4f2d-ad01-df32ad96807b req-79b3af43-f9a8-40fc-bcde-5934b89457b9 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0889554 May 20 04:23:57.232711 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6139ae73-12e3-4f2d-ad01-df32ad96807b req-d5e0d0ee-5bda-44cb-8c83-0a39db9e27c1 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0965877 May 20 04:23:57.316483 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 36278) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:57.333792 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-be054bc4-7ef1-41e0-8770-4ee91ef821d6 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:57.349050 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-be054bc4-7ef1-41e0-8770-4ee91ef821d6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:23:57.352925 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-be054bc4-7ef1-41e0-8770-4ee91ef821d6 admin admin] Usage tracker for resource:security_group_rule and tenant:76151d9dad154cb78207f30cfe2f2ef4 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:23:57.355727 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-be054bc4-7ef1-41e0-8770-4ee91ef821d6 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:23:57.361244 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-be054bc4-7ef1-41e0-8770-4ee91ef821d6 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:23:57.444912 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-be054bc4-7ef1-41e0-8770-4ee91ef821d6 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:57.449994 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-be054bc4-7ef1-41e0-8770-4ee91ef821d6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:57.458548 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-b03d7900-1588-488b-9abd-4c665f8c0427 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:57.499311 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-be054bc4-7ef1-41e0-8770-4ee91ef821d6 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=76151d9dad154cb78207f30cfe2f2ef4&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1816404 May 20 04:23:57.500874 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-be054bc4-7ef1-41e0-8770-4ee91ef821d6 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=fe5396fc-b2eb-4694-9843-8fc503b45f2c,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:57.502204 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 36286) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:57.504038 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-be054bc4-7ef1-41e0-8770-4ee91ef821d6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:57.568686 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-84f06910-0f20-4ff3-9726-517986e2fe73 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:57.595664 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-84f06910-0f20-4ff3-9726-517986e2fe73 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:57.649082 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-84f06910-0f20-4ff3-9726-517986e2fe73 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:57.654033 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-4d6f54e9-ec7d-4c97-adcd-184fc0c47ed1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:57.665628 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-84f06910-0f20-4ff3-9726-517986e2fe73 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=fe5396fc-b2eb-4694-9843-8fc503b45f2c,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:57.669336 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-84f06910-0f20-4ff3-9726-517986e2fe73 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:57.674594 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-84f06910-0f20-4ff3-9726-517986e2fe73 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:57.675715 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-84f06910-0f20-4ff3-9726-517986e2fe73 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/fe5396fc-b2eb-4694-9843-8fc503b45f2c HTTP/1.1" status: 204 len: 168 time: 0.1716447 May 20 04:23:57.747756 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-386cfaf2-53c0-4fd6-805f-c46a1b033cf2 req-b1478448-ec37-4ba2-bae8-420002284525 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:57.749183 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-386cfaf2-53c0-4fd6-805f-c46a1b033cf2 req-b1478448-ec37-4ba2-bae8-420002284525 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.0958569 May 20 04:23:57.793514 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-386cfaf2-53c0-4fd6-805f-c46a1b033cf2 req-76f6e4ef-b895-40b6-ae30-e9816a8f83f1 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0385017 May 20 04:23:57.971287 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2c1d83e8-32bf-4e21-bdea-143bd64ed11e req-8b805635-bb96-4f0c-94d3-cc1fa21124b7 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:57.972744 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2c1d83e8-32bf-4e21-bdea-143bd64ed11e req-8b805635-bb96-4f0c-94d3-cc1fa21124b7 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 945 time: 0.0825028 May 20 04:23:58.017410 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2c1d83e8-32bf-4e21-bdea-143bd64ed11e req-67260661-684b-45a7-9b29-9a70bf346c13 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0384824 May 20 04:23:58.422911 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cfa5b4f0-2d4a-421d-a113-04292f0d1085 req-a1cf3e36-4a21-40e5-aa56-32717398c657 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:58.424397 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cfa5b4f0-2d4a-421d-a113-04292f0d1085 req-a1cf3e36-4a21-40e5-aa56-32717398c657 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0783989 May 20 04:23:58.470244 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cfa5b4f0-2d4a-421d-a113-04292f0d1085 req-3eda5127-277c-491a-96fe-805c185dd619 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0395362 May 20 04:23:58.851527 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 36332) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:58.972694 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-70a426a0-eac4-4b93-a3ba-3d58a1ee6187 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:58.981092 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-397b1f31-b9ff-4005-9b32-285377a20908 req-1577a5bc-4a2d-4efa-bc9b-c11cb7a64b0d tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:58.982665 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-397b1f31-b9ff-4005-9b32-285377a20908 req-1577a5bc-4a2d-4efa-bc9b-c11cb7a64b0d tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.0824285 May 20 04:23:58.987989 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-70a426a0-eac4-4b93-a3ba-3d58a1ee6187 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:23:58.992178 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-70a426a0-eac4-4b93-a3ba-3d58a1ee6187 admin admin] Usage tracker for resource:security_group_rule and tenant:f8dbca50fd584a35b621b4d0f20e066e is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:23:58.995073 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-70a426a0-eac4-4b93-a3ba-3d58a1ee6187 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:23:59.000821 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-70a426a0-eac4-4b93-a3ba-3d58a1ee6187 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:23:59.027504 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-397b1f31-b9ff-4005-9b32-285377a20908 req-9f0c509b-8163-46db-b5d4-d45f860412b8 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0389953 May 20 04:23:59.094724 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-70a426a0-eac4-4b93-a3ba-3d58a1ee6187 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:59.099607 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-70a426a0-eac4-4b93-a3ba-3d58a1ee6187 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:59.106768 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:59.147008 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-70a426a0-eac4-4b93-a3ba-3d58a1ee6187 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=f8dbca50fd584a35b621b4d0f20e066e&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2944400 May 20 04:23:59.147855 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-70a426a0-eac4-4b93-a3ba-3d58a1ee6187 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=6bbbf8e9-1c2a-4d7b-a03b-3a9039e65544,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:59.150067 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 36340) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:23:59.152717 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-70a426a0-eac4-4b93-a3ba-3d58a1ee6187 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:59.207518 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ae6a4131-2f52-4044-af50-f174d2bbd876 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:59.210754 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-facadd6b-e3cf-465c-88b8-3550da1e80a2 req-1a1f174f-0fb4-4b83-84ab-5f68358a31f9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:59.212266 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-facadd6b-e3cf-465c-88b8-3550da1e80a2 req-1a1f174f-0fb4-4b83-84ab-5f68358a31f9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 945 time: 0.0818658 May 20 04:23:59.233030 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ae6a4131-2f52-4044-af50-f174d2bbd876 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:59.260708 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-facadd6b-e3cf-465c-88b8-3550da1e80a2 req-c70b7103-7afe-496f-8dbd-81cab142ef54 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0422029 May 20 04:23:59.283797 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ae6a4131-2f52-4044-af50-f174d2bbd876 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:59.287669 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d22f1933-748d-4089-8fb3-332ddc6447c2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:23:59.298686 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ae6a4131-2f52-4044-af50-f174d2bbd876 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=6bbbf8e9-1c2a-4d7b-a03b-3a9039e65544,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:23:59.302977 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-ae6a4131-2f52-4044-af50-f174d2bbd876 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:23:59.305033 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ae6a4131-2f52-4044-af50-f174d2bbd876 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:23:59.306083 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-ae6a4131-2f52-4044-af50-f174d2bbd876 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/6bbbf8e9-1c2a-4d7b-a03b-3a9039e65544 HTTP/1.1" status: 204 len: 168 time: 0.1549430 May 20 04:23:59.676184 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4bc7d57a-d185-4b66-9376-f056adf90e7e req-4ccb4701-3a97-4178-86ca-8ee7fbcdc741 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:23:59.677975 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4bc7d57a-d185-4b66-9376-f056adf90e7e req-4ccb4701-3a97-4178-86ca-8ee7fbcdc741 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0947165 May 20 04:23:59.728580 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4bc7d57a-d185-4b66-9376-f056adf90e7e req-c6d3b28b-189b-4725-b027-cbe0ca4efe2c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0434511 May 20 04:24:00.227618 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c9d012a3-c7d9-4dfa-a9af-3195ddc7eb65 req-efa5d94d-6f26-409b-a6ae-7464d1304f38 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:00.229067 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c9d012a3-c7d9-4dfa-a9af-3195ddc7eb65 req-efa5d94d-6f26-409b-a6ae-7464d1304f38 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.0849085 May 20 04:24:00.274220 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c9d012a3-c7d9-4dfa-a9af-3195ddc7eb65 req-0a18721e-5b37-4306-80d2-eeacc90ad146 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0390215 May 20 04:24:00.502129 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e931a399-362b-40da-af4d-90cc4d54e5c0 req-e6233478-004d-4b86-9761-bdbe64437137 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:00.504084 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e931a399-362b-40da-af4d-90cc4d54e5c0 req-e6233478-004d-4b86-9761-bdbe64437137 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 945 time: 0.0985708 May 20 04:24:00.536612 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 36410) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:00.572087 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e931a399-362b-40da-af4d-90cc4d54e5c0 req-695fa80e-2607-4fb3-9eb1-1a1c0093723b tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0610545 May 20 04:24:00.701506 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-595032b4-d954-4afa-94b4-d5f6e21fc4d9 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:00.785644 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-595032b4-d954-4afa-94b4-d5f6e21fc4d9 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:00.789557 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-595032b4-d954-4afa-94b4-d5f6e21fc4d9 admin admin] Usage tracker for resource:security_group_rule and tenant:3dc874ba2d794a4fbd321a643d6ff325 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:00.792356 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-595032b4-d954-4afa-94b4-d5f6e21fc4d9 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:00.797932 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-595032b4-d954-4afa-94b4-d5f6e21fc4d9 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:00.883991 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-595032b4-d954-4afa-94b4-d5f6e21fc4d9 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:00.888870 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-595032b4-d954-4afa-94b4-d5f6e21fc4d9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:00.897605 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-5a3732e2-50c2-4016-8672-c94f27a8b7cc admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:00.924547 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0ba97b7-8202-4613-a651-4243ccda120e req-7e34e26e-0e90-4405-ac79-9fc637a90ac9 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:00.925979 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d0ba97b7-8202-4613-a651-4243ccda120e req-7e34e26e-0e90-4405-ac79-9fc637a90ac9 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0849779 May 20 04:24:00.936768 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-595032b4-d954-4afa-94b4-d5f6e21fc4d9 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=3dc874ba2d794a4fbd321a643d6ff325&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3973989 May 20 04:24:00.938473 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-595032b4-d954-4afa-94b4-d5f6e21fc4d9 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=bdec6ffe-995d-4209-8ad4-ca6a2a3abddb,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:00.940833 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 36424) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:00.952321 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-595032b4-d954-4afa-94b4-d5f6e21fc4d9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:00.973681 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d0ba97b7-8202-4613-a651-4243ccda120e req-ddb3dcb0-119c-484a-9d32-d8558ca98981 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0412984 May 20 04:24:01.002027 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-02bbc84d-579e-478a-993e-03c846553b9b admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:01.028111 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-02bbc84d-579e-478a-993e-03c846553b9b admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:01.085215 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-02bbc84d-579e-478a-993e-03c846553b9b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:01.088711 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-8d139106-40c5-4d89-a0d0-3d455676bd4b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:01.101616 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-02bbc84d-579e-478a-993e-03c846553b9b admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=bdec6ffe-995d-4209-8ad4-ca6a2a3abddb,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:01.104496 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-02bbc84d-579e-478a-993e-03c846553b9b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:01.106427 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-02bbc84d-579e-478a-993e-03c846553b9b admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:01.107972 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-02bbc84d-579e-478a-993e-03c846553b9b admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/bdec6ffe-995d-4209-8ad4-ca6a2a3abddb HTTP/1.1" status: 204 len: 168 time: 0.1658475 May 20 04:24:01.477041 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-488df4dd-5ce5-4ab8-bff2-94e250a1630e req-b6f7982e-d2ec-4b27-b70f-351226c55ee6 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:01.478384 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-488df4dd-5ce5-4ab8-bff2-94e250a1630e req-b6f7982e-d2ec-4b27-b70f-351226c55ee6 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.0941987 May 20 04:24:01.529653 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-488df4dd-5ce5-4ab8-bff2-94e250a1630e req-1ba98c44-ba56-4e7d-98d0-54991baf9945 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0419297 May 20 04:24:01.753606 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-302f9b7d-a39c-4692-8737-840d24e837b7 req-55d51d54-cba5-49f7-a5b0-1eb264cc911f tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:01.754956 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-302f9b7d-a39c-4692-8737-840d24e837b7 req-55d51d54-cba5-49f7-a5b0-1eb264cc911f tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 945 time: 0.0842979 May 20 04:24:01.799777 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-302f9b7d-a39c-4692-8737-840d24e837b7 req-b266e65f-b78c-4042-a010-39537edfd47a tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0390742 May 20 04:24:02.192640 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-986891f7-8204-46ac-b3d5-b1420ab64e51 req-75bce1a5-3315-472f-8b53-8b98c9b91d9f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:02.194091 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-986891f7-8204-46ac-b3d5-b1420ab64e51 req-75bce1a5-3315-472f-8b53-8b98c9b91d9f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0972917 May 20 04:24:02.237814 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-986891f7-8204-46ac-b3d5-b1420ab64e51 req-9bc27c53-dcfb-474b-8017-fa90a5e3fb07 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0379491 May 20 04:24:02.733225 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-43508b41-76b3-486a-a999-6d8152a07048 req-f9d6449a-c320-45fd-8c60-514260ad520b tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:02.734746 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-43508b41-76b3-486a-a999-6d8152a07048 req-f9d6449a-c320-45fd-8c60-514260ad520b tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.0867784 May 20 04:24:02.780793 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-43508b41-76b3-486a-a999-6d8152a07048 req-aa06c33a-8ab5-4a9d-a15c-25affd84d20f tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0400422 May 20 04:24:02.994419 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4e8ab235-7791-4134-a4f4-3d8bcdba9cb8 req-7dce14a2-0890-4202-91f8-3121d2031f34 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:02.995954 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4e8ab235-7791-4134-a4f4-3d8bcdba9cb8 req-7dce14a2-0890-4202-91f8-3121d2031f34 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 945 time: 0.0873940 May 20 04:24:03.042461 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4e8ab235-7791-4134-a4f4-3d8bcdba9cb8 req-73fecf33-b276-4cec-a2f1-3e520df51d3e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0399179 May 20 04:24:03.438496 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dd464e11-1812-44ec-a678-76cc6140fd16 req-0a3c60a8-f87e-4c3e-89b0-c87c982791a4 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:03.439848 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-dd464e11-1812-44ec-a678-76cc6140fd16 req-0a3c60a8-f87e-4c3e-89b0-c87c982791a4 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0823603 May 20 04:24:03.490490 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-dd464e11-1812-44ec-a678-76cc6140fd16 req-d8098b9f-c11c-4463-b1b0-0afb79d93d6c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0423694 May 20 04:24:03.977656 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9f8a6a4f-45dd-48bc-a8ad-041e3eaac9a5 req-5cc162a9-a670-4a0c-b013-687d855a9774 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:03.979362 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9f8a6a4f-45dd-48bc-a8ad-041e3eaac9a5 req-5cc162a9-a670-4a0c-b013-687d855a9774 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.0850248 May 20 04:24:04.025638 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9f8a6a4f-45dd-48bc-a8ad-041e3eaac9a5 req-cd61e808-309e-4fbc-8409-bafe92efd8d2 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0395639 May 20 04:24:04.295673 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1af4ccc4-a06f-4874-b906-8eb505f4c1ec req-47f06603-1315-4c49-bf21-cd379147153b tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:04.297123 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1af4ccc4-a06f-4874-b906-8eb505f4c1ec req-47f06603-1315-4c49-bf21-cd379147153b tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 945 time: 0.1393940 May 20 04:24:04.358150 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1af4ccc4-a06f-4874-b906-8eb505f4c1ec req-4665d9cb-d554-480d-8a16-cbf5857e6952 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0529349 May 20 04:24:04.701177 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-40dfa5f8-9bae-4c54-b90f-26dc4dc22ed3 req-c127c0cf-afe6-4cd9-855a-ee981af34ce5 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:04.702612 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-40dfa5f8-9bae-4c54-b90f-26dc4dc22ed3 req-c127c0cf-afe6-4cd9-855a-ee981af34ce5 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0878904 May 20 04:24:04.752527 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-40dfa5f8-9bae-4c54-b90f-26dc4dc22ed3 req-8fe4e135-c668-494f-ae2b-deaeba97db65 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0430210 May 20 04:24:05.216529 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7eacb71c-552f-453a-942c-0cae43e68c31 req-9fb51e83-2cd7-41ec-a81d-eb06ed124ba6 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:05.218280 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7eacb71c-552f-453a-942c-0cae43e68c31 req-9fb51e83-2cd7-41ec-a81d-eb06ed124ba6 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.0832827 May 20 04:24:05.262628 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7eacb71c-552f-453a-942c-0cae43e68c31 req-aea1abf3-28dd-41d0-8b7c-10116fb8a48e tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0384159 May 20 04:24:05.549089 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:05.575381 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9653fc99-194d-45d3-98d7-9c37bbc7e791 req-aa951f8d-0f0a-4453-9a1f-ed2ef7d25daa tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:05.577175 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9653fc99-194d-45d3-98d7-9c37bbc7e791 req-aa951f8d-0f0a-4453-9a1f-ed2ef7d25daa tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 945 time: 0.1009510 May 20 04:24:05.623422 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9653fc99-194d-45d3-98d7-9c37bbc7e791 req-860dee05-f590-4e72-985c-973d4d556198 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0397155 May 20 04:24:05.951320 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-be6022de-cb01-4ec3-b22f-a16cb3c42d6f req-297f50e5-afe4-4d21-a31b-5e1b4438ebe3 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:05.951772 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-be6022de-cb01-4ec3-b22f-a16cb3c42d6f req-297f50e5-afe4-4d21-a31b-5e1b4438ebe3 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0855944 May 20 04:24:05.997227 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-be6022de-cb01-4ec3-b22f-a16cb3c42d6f req-bedd6e86-9731-41a5-89fe-6943c3d61bfe tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0396602 May 20 04:24:06.462151 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c9041d54-a4e0-4fe5-b775-8566af38f879 req-a945228b-1ede-4bcb-9d1f-71f513d4d5b0 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:06.463557 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c9041d54-a4e0-4fe5-b775-8566af38f879 req-a945228b-1ede-4bcb-9d1f-71f513d4d5b0 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.0956104 May 20 04:24:06.512652 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c9041d54-a4e0-4fe5-b775-8566af38f879 req-b98fed34-d53a-4a85-8163-32540d6a49f7 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0413129 May 20 04:24:06.829302 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c3eaf765-590a-475d-8b0c-8b759f8268da req-68257aa3-89d7-47af-9e5d-af6572e5dfb6 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:06.830745 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c3eaf765-590a-475d-8b0c-8b759f8268da req-68257aa3-89d7-47af-9e5d-af6572e5dfb6 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 945 time: 0.0835440 May 20 04:24:06.875155 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c3eaf765-590a-475d-8b0c-8b759f8268da req-c4d0a658-49f7-4034-b696-d0210ea402c3 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0386405 May 20 04:24:07.187639 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7e4f2385-8aa3-4929-a2d2-ba211e344572 req-a35f1d08-fdb5-467c-a713-1dea28d31e1a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:07.189221 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7e4f2385-8aa3-4929-a2d2-ba211e344572 req-a35f1d08-fdb5-467c-a713-1dea28d31e1a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0796762 May 20 04:24:07.233146 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7e4f2385-8aa3-4929-a2d2-ba211e344572 req-31fe558e-1e92-43e4-91e8-d3bdd1e8acd5 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0377965 May 20 04:24:07.706993 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c400d844-13af-4be3-9cb3-74e3ce324d88 req-5811a8e6-b99d-4cac-b117-e1e85610ecd3 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:07.708704 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c400d844-13af-4be3-9cb3-74e3ce324d88 req-5811a8e6-b99d-4cac-b117-e1e85610ecd3 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.0848341 May 20 04:24:07.755293 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c400d844-13af-4be3-9cb3-74e3ce324d88 req-e258fdf8-4569-4dd1-b395-e7c0bb2488d8 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0404711 May 20 04:24:08.066755 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1390adbb-0037-47f3-ba4c-2d85a630b775 req-4de8e6e8-c042-46ae-9eff-39efabce3ad3 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:08.066899 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1390adbb-0037-47f3-ba4c-2d85a630b775 req-4de8e6e8-c042-46ae-9eff-39efabce3ad3 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 945 time: 0.0828512 May 20 04:24:08.120139 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1390adbb-0037-47f3-ba4c-2d85a630b775 req-7ca55348-d94c-4cf1-89bb-081649d6724e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0464559 May 20 04:24:08.488086 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d68fe7e5-b0fb-4e86-a57b-e858bc34f92b req-2e429f2e-281d-4e22-a084-99aa22df4fcb tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:08.489665 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d68fe7e5-b0fb-4e86-a57b-e858bc34f92b req-2e429f2e-281d-4e22-a084-99aa22df4fcb tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.1464767 May 20 04:24:08.527287 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b3dc5a2-e734-4f1a-b802-fdbe335560df req-4f2206db-5c0f-47c4-aa3d-b7559b4df1a3 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:08.528726 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0b3dc5a2-e734-4f1a-b802-fdbe335560df req-4f2206db-5c0f-47c4-aa3d-b7559b4df1a3 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.74 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.1252098 May 20 04:24:08.593750 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d68fe7e5-b0fb-4e86-a57b-e858bc34f92b req-08fc360f-6710-4765-a1d4-76840f98b31b tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0976906 May 20 04:24:08.677875 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-0b3dc5a2-e734-4f1a-b802-fdbe335560df req-04de94b9-aee2-4cc5-a626-a6072e5e5018 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Deleting port efb81444-f373-4e9c-a0aa-e3270bd4b980 {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:24:08.678149 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-0b3dc5a2-e734-4f1a-b802-fdbe335560df req-04de94b9-aee2-4cc5-a626-a6072e5e5018 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:08.812844 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-0b3dc5a2-e734-4f1a-b802-fdbe335560df req-04de94b9-aee2-4cc5-a626-a6072e5e5018 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] For port efb81444-f373-4e9c-a0aa-e3270bd4b980, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=efb81444-f373-4e9c-a0aa-e3270bd4b980,segment=NetworkSegment(3c4a2238-8a32-4d92-a1b1-8af91c1c27a7),segment_id=3c4a2238-8a32-4d92-a1b1-8af91c1c27a7)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:24:08.813583 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-0b3dc5a2-e734-4f1a-b802-fdbe335560df req-04de94b9-aee2-4cc5-a626-a6072e5e5018 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:08.931943 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-0b3dc5a2-e734-4f1a-b802-fdbe335560df req-04de94b9-aee2-4cc5-a626-a6072e5e5018 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Calling delete_port for efb81444-f373-4e9c-a0aa-e3270bd4b980 owned by compute:nova {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:24:09.152596 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9ce1a2c5-85a0-4616-8cdf-fad327d98bcb req-c117c8cd-c5c1-472f-baf9-ec8661fdc724 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:09.154013 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9ce1a2c5-85a0-4616-8cdf-fad327d98bcb req-c117c8cd-c5c1-472f-baf9-ec8661fdc724 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 945 time: 0.2826037 May 20 04:24:09.243375 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9ce1a2c5-85a0-4616-8cdf-fad327d98bcb req-91b027c4-5c09-4b4d-93ea-dce9765de84b tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/security-groups?id=06880d35-d748-4b5a-88d1-1192a88678c7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0835052 May 20 04:24:09.252977 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-0b3dc5a2-e734-4f1a-b802-fdbe335560df req-04de94b9-aee2-4cc5-a626-a6072e5e5018 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:09.260659 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9933dbaf-f2d3-4019-a95a-bfc8f0285605 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:09.305141 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-04de94b9-aee2-4cc5-a626-a6072e5e5018 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Pushing event deleted for resources: {'Port': ['ID=efb81444-f373-4e9c-a0aa-e3270bd4b980,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:09.334068 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-04de94b9-aee2-4cc5-a626-a6072e5e5018 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:09.378997 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [req-0b3dc5a2-e734-4f1a-b802-fdbe335560df req-04de94b9-aee2-4cc5-a626-a6072e5e5018 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] No DVR routers for this DVR port efb81444-f373-4e9c-a0aa-e3270bd4b980 on host hv-cinder-78240 {{(pid=91643) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:24:09.382190 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-0b3dc5a2-e734-4f1a-b802-fdbe335560df req-04de94b9-aee2-4cc5-a626-a6072e5e5018 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:09.384635 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'bea4be17-559d-407e-9b61-ff2cb94317dc', 'name': 'network-vif-deleted', 'tag': 'efb81444-f373-4e9c-a0aa-e3270bd4b980'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:24:09.397229 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0b3dc5a2-e734-4f1a-b802-fdbe335560df req-04de94b9-aee2-4cc5-a626-a6072e5e5018 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.74 "DELETE /v2.0/ports/efb81444-f373-4e9c-a0aa-e3270bd4b980 HTTP/1.1" status: 204 len: 173 time: 0.8599789 May 20 04:24:09.430106 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'bea4be17-559d-407e-9b61-ff2cb94317dc', 'name': 'network-vif-deleted', 'tag': 'efb81444-f373-4e9c-a0aa-e3270bd4b980', 'status': 'completed', 'code': 200} May 20 04:24:09.609850 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['efb81444-f373-4e9c-a0aa-e3270bd4b980'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:24:09.800355 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-089aa965-be4b-4b97-8243-8f390483e167 req-223fa954-4449-4429-9d2d-295187ba2f4f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:09.802354 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-089aa965-be4b-4b97-8243-8f390483e167 req-223fa954-4449-4429-9d2d-295187ba2f4f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0874264 May 20 04:24:09.852839 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-089aa965-be4b-4b97-8243-8f390483e167 req-27bde9f9-c473-4465-8649-f16a411ae129 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0444067 May 20 04:24:10.403394 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-e9851f60-fc11-493f-bfc4-268663c70bc2 req-c35cf336-7feb-4b7d-a18d-19b7718fd933 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Synchronizing usage tracker for tenant:0b29d684ade544028fa17828cde46d56 on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:24:10.428355 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-e9851f60-fc11-493f-bfc4-268663c70bc2 req-c35cf336-7feb-4b7d-a18d-19b7718fd933 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Unset dirty status for tenant:0b29d684ade544028fa17828cde46d56 on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:24:10.428355 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e9851f60-fc11-493f-bfc4-268663c70bc2 req-c35cf336-7feb-4b7d-a18d-19b7718fd933 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 210 time: 0.0608568 May 20 04:24:11.053615 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5afe2e60-b162-48fb-b2dc-c7bf9884d381 req-56988585-1e09-4639-8711-34dcf97f0d37 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:11.055125 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5afe2e60-b162-48fb-b2dc-c7bf9884d381 req-56988585-1e09-4639-8711-34dcf97f0d37 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0870605 May 20 04:24:11.110141 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5afe2e60-b162-48fb-b2dc-c7bf9884d381 req-666c6a1b-c6fa-41cf-b1af-75cb86dbd501 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0487111 May 20 04:24:11.576881 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fb734694-5329-4b2b-98ee-bb607b619aa6 req-c34f7189-65c2-4027-8fbf-51d1346c3517 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 210 time: 0.0420246 May 20 04:24:12.307891 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-22f28c15-f98c-4944-b4a4-fa35ba5eab9f req-4c077d39-26d8-4e08-b182-1da608f02783 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:12.309418 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-22f28c15-f98c-4944-b4a4-fa35ba5eab9f req-4c077d39-26d8-4e08-b182-1da608f02783 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0889590 May 20 04:24:12.360031 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-22f28c15-f98c-4944-b4a4-fa35ba5eab9f req-1a2a50b7-5e18-487b-9c19-47bb1c547364 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0427043 May 20 04:24:12.595241 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-25de1269-b5a8-464b-aba1-7b32ef485404 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:12.725377 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-56bb85b7-5a43-4709-b1cc-5a581a5cf454 req-4422b827-3e98-4000-ab1c-9ccc2f7e819b tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 210 time: 0.0423505 May 20 04:24:13.566793 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-51451749-0684-4be5-ae1b-de3329200266 req-c3209f8e-93d0-4b4a-987a-0c9bb8cac116 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:13.568368 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-51451749-0684-4be5-ae1b-de3329200266 req-c3209f8e-93d0-4b4a-987a-0c9bb8cac116 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0851645 May 20 04:24:13.613146 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-51451749-0684-4be5-ae1b-de3329200266 req-69186a8e-1307-4316-95ee-232f82f80dc3 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0390177 May 20 04:24:13.885520 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-14c15985-084a-4745-a6e5-71d1125f8da5 req-910dcf7b-f8aa-458a-bb88-217faf9356fc tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 210 time: 0.0449939 May 20 04:24:14.803548 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0294f1e-2f0e-4e20-9292-07c19148f201 req-4a6a2991-5126-422b-a304-356c5dec6c28 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:14.805041 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b0294f1e-2f0e-4e20-9292-07c19148f201 req-4a6a2991-5126-422b-a304-356c5dec6c28 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0799880 May 20 04:24:14.849701 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b0294f1e-2f0e-4e20-9292-07c19148f201 req-3971f854-d7c9-4423-ac42-95340548b48a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0388944 May 20 04:24:14.958435 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:15.039173 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-86f4e763-56aa-445d-ac8b-861787b64d17 req-9b17a4bd-b959-4b24-a706-ac31e084aac9 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 210 time: 0.0456924 May 20 04:24:15.078106 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-efba0391-ec19-4647-95b6-d492a4699213 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:15.317537 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device efb81444-f373-4e9c-a0aa-e3270bd4b980 no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:24:15.369828 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: efb81444-f373-4e9c-a0aa-e3270bd4b980 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:24:15.370169 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device efb81444-f373-4e9c-a0aa-e3270bd4b980 not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:24:15.850734 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ac4b5fa2-cc6f-4cfc-87da-b3b8e2e93f02 req-6e2dd263-e858-4899-af7e-12cc3a1b1905 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0775201 May 20 04:24:15.947676 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ac4b5fa2-cc6f-4cfc-87da-b3b8e2e93f02 req-1fc19889-3922-4a73-9bcb-5bd840576f23 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:15.949120 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ac4b5fa2-cc6f-4cfc-87da-b3b8e2e93f02 req-1fc19889-3922-4a73-9bcb-5bd840576f23 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=9fafdf3a29024ac588e5b24719735677&device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 1098 time: 0.0764692 May 20 04:24:16.143879 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ac4b5fa2-cc6f-4cfc-87da-b3b8e2e93f02 req-9e1b5b91-61ec-446d-bbc5-ab469649737d service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:16.145386 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ac4b5fa2-cc6f-4cfc-87da-b3b8e2e93f02 req-9e1b5b91-61ec-446d-bbc5-ab469649737d service neutron] 10.222.0.74 "GET /v2.0/networks?id=64bcea8a-6d35-406b-a823-fa1185008174 HTTP/1.1" status: 200 len: 918 time: 0.1500714 May 20 04:24:16.158342 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b868a5db-5277-400f-a6bf-0176843e7259 req-1b7d220f-03e6-46fd-8ddc-38988415c29a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:16.159911 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b868a5db-5277-400f-a6bf-0176843e7259 req-1b7d220f-03e6-46fd-8ddc-38988415c29a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.2017450 May 20 04:24:16.168692 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:16.361963 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f51a3575-d1ac-44d3-99e5-39bac55d1125 req-aafbba99-0596-41c9-8234-a746e0ce39cb tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] 10.222.0.39 "GET /v2.0/ports?device_id=bea4be17-559d-407e-9b61-ff2cb94317dc HTTP/1.1" status: 200 len: 210 time: 0.1998677 May 20 04:24:16.396949 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b868a5db-5277-400f-a6bf-0176843e7259 req-c9273e6b-fe47-4d2d-94db-150841220b97 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1489737 May 20 04:24:16.400904 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ac4b5fa2-cc6f-4cfc-87da-b3b8e2e93f02 req-e965e524-cb5d-4e1c-a945-84545de865f8 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=535d5637-ae66-43cb-bd16-3543bd3b7a8b HTTP/1.1" status: 200 len: 1001 time: 0.2400699 May 20 04:24:16.469511 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ac4b5fa2-cc6f-4cfc-87da-b3b8e2e93f02 req-634882d2-0066-453e-a256-164161086521 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:16.472325 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ac4b5fa2-cc6f-4cfc-87da-b3b8e2e93f02 req-634882d2-0066-453e-a256-164161086521 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=203a3962-2282-4f51-ac92-c9695b5c0cfe HTTP/1.1" status: 200 len: 856 time: 0.0632513 May 20 04:24:16.554102 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ac4b5fa2-cc6f-4cfc-87da-b3b8e2e93f02 req-4ba9fcdb-074c-432e-82d9-f479e3386a84 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:16.555603 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ac4b5fa2-cc6f-4cfc-87da-b3b8e2e93f02 req-4ba9fcdb-074c-432e-82d9-f479e3386a84 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=64bcea8a-6d35-406b-a823-fa1185008174&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0774567 May 20 04:24:16.710784 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ac4b5fa2-cc6f-4cfc-87da-b3b8e2e93f02 req-22f14106-db5e-4b45-9a90-00aaf769b3c8 service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1430714 May 20 04:24:16.846799 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ac4b5fa2-cc6f-4cfc-87da-b3b8e2e93f02 req-751dc468-161c-4828-8a67-2da1f802471e service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1280563 May 20 04:24:17.236999 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-daac93ee-58b9-49d7-b3ec-d0e2221da647 req-9dfa902e-cecd-4aed-89de-13c28efa6178 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:17.238550 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-daac93ee-58b9-49d7-b3ec-d0e2221da647 req-9dfa902e-cecd-4aed-89de-13c28efa6178 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 1099 time: 0.0988770 May 20 04:24:17.289428 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 36636) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:17.418226 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1843ae6f-1d1f-49e0-bf59-aa2560cc247d admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:17.436351 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-1843ae6f-1d1f-49e0-bf59-aa2560cc247d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:17.440494 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-1843ae6f-1d1f-49e0-bf59-aa2560cc247d admin admin] Usage tracker for resource:security_group_rule and tenant:9798b1b70fc44199ab66c58d2b0e4837 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:17.443487 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-1843ae6f-1d1f-49e0-bf59-aa2560cc247d admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:17.449291 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-1843ae6f-1d1f-49e0-bf59-aa2560cc247d admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:17.535275 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1843ae6f-1d1f-49e0-bf59-aa2560cc247d admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:17.540522 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1843ae6f-1d1f-49e0-bf59-aa2560cc247d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:17.573474 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-31ab0065-c8e8-4888-a4ca-f153dfe60850 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:17.593898 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-1843ae6f-1d1f-49e0-bf59-aa2560cc247d admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=9798b1b70fc44199ab66c58d2b0e4837&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3033633 May 20 04:24:17.594924 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 36652) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:17.609035 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1843ae6f-1d1f-49e0-bf59-aa2560cc247d admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=86c48db6-1657-4079-8f50-61bab8d6a63c,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:17.612713 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-1843ae6f-1d1f-49e0-bf59-aa2560cc247d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.039s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:17.660583 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eecf71d9-1a73-4c12-be44-cf290947ecc1 req-72ac00a0-e7df-49ff-b529-c29cd3da5707 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:17.662051 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-eecf71d9-1a73-4c12-be44-cf290947ecc1 req-72ac00a0-e7df-49ff-b529-c29cd3da5707 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.1423340 May 20 04:24:17.695063 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-ad1194b0-df1b-4a95-9763-6e3a03418c59 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:17.736786 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-eecf71d9-1a73-4c12-be44-cf290947ecc1 req-2c05e2c2-539c-49c2-ad9d-849121008f26 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0648308 May 20 04:24:17.738591 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-ad1194b0-df1b-4a95-9763-6e3a03418c59 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:17.809947 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:17.811493 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-ad1194b0-df1b-4a95-9763-6e3a03418c59 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:17.816447 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:17.828676 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ad1194b0-df1b-4a95-9763-6e3a03418c59 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=86c48db6-1657-4079-8f50-61bab8d6a63c,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:17.833379 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-ad1194b0-df1b-4a95-9763-6e3a03418c59 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:17.836843 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-ad1194b0-df1b-4a95-9763-6e3a03418c59 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/86c48db6-1657-4079-8f50-61bab8d6a63c HTTP/1.1" status: 204 len: 168 time: 0.2385643 May 20 04:24:17.836843 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-ad1194b0-df1b-4a95-9763-6e3a03418c59 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:18.957180 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-40d3074a-91e5-4a0e-a8c2-1b781f17c72f req-117e1b32-722e-4670-acbe-a8d4da6ac11a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:18.958703 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-40d3074a-91e5-4a0e-a8c2-1b781f17c72f req-117e1b32-722e-4670-acbe-a8d4da6ac11a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0899241 May 20 04:24:19.010685 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-40d3074a-91e5-4a0e-a8c2-1b781f17c72f req-781fb549-074e-4233-8088-bde2fd5492d9 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0460978 May 20 04:24:19.560078 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-daac93ee-58b9-49d7-b3ec-d0e2221da647 req-886153f6-c623-4660-a9b3-ff98a7d2f22a service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0439427 May 20 04:24:20.209824 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-47e1acc0-ff26-4b35-95cb-13ee175d3131 req-d178eb9a-f9ca-4fbd-a1e5-b421f53ba6d2 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:20.211170 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-47e1acc0-ff26-4b35-95cb-13ee175d3131 req-d178eb9a-f9ca-4fbd-a1e5-b421f53ba6d2 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0896192 May 20 04:24:20.260402 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-47e1acc0-ff26-4b35-95cb-13ee175d3131 req-b4e46d87-2453-43f1-a2c6-cffccc463ec4 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0434077 May 20 04:24:21.454720 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-96edcdac-ba35-4955-97fb-25d8c4172e8c req-84a7761b-6623-4483-8da1-5c916a43b4f1 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:21.456186 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-96edcdac-ba35-4955-97fb-25d8c4172e8c req-84a7761b-6623-4483-8da1-5c916a43b4f1 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0798872 May 20 04:24:21.500171 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-96edcdac-ba35-4955-97fb-25d8c4172e8c req-06e36ed5-653c-41c3-a025-9218d12560d3 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0382302 May 20 04:24:22.697273 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-223cda32-9356-4c7b-9a23-19e86731f71d req-052f8696-614b-4903-a607-e9371723c381 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:22.698731 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-223cda32-9356-4c7b-9a23-19e86731f71d req-052f8696-614b-4903-a607-e9371723c381 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0907717 May 20 04:24:22.742961 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-223cda32-9356-4c7b-9a23-19e86731f71d req-ae1f32e9-2b34-4d27-9932-01649b3ec19a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0383184 May 20 04:24:23.936339 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-74e7cc67-0a9e-4759-97a8-fff216f7afe7 req-7c3f88f3-6da8-415f-aee7-5e24527aa75c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:23.937766 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-74e7cc67-0a9e-4759-97a8-fff216f7afe7 req-7c3f88f3-6da8-415f-aee7-5e24527aa75c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0916553 May 20 04:24:23.986879 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-74e7cc67-0a9e-4759-97a8-fff216f7afe7 req-cf607fd6-5d6e-435c-96ae-b4a6aac8065d tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0429711 May 20 04:24:24.794629 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-786c7605-96a8-4b68-9576-762932e6f555 req-3867a62e-46b6-4c18-bef7-256a3d609972 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:24.796075 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-786c7605-96a8-4b68-9576-762932e6f555 req-3867a62e-46b6-4c18-bef7-256a3d609972 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/networks?id=54fc8d38-193b-49c4-b873-c5353723791f HTTP/1.1" status: 200 len: 815 time: 0.1547489 May 20 04:24:24.811400 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-786c7605-96a8-4b68-9576-762932e6f555 req-9bbd4f74-f7be-484c-918a-a636d955839f tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/quotas/51fcbc78284541db832e4780e76158af HTTP/1.1" status: 200 len: 375 time: 0.0097160 May 20 04:24:24.897915 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-786c7605-96a8-4b68-9576-762932e6f555 req-b41d2b85-00bf-4aef-acf2-f0d06f6af6f2 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=51fcbc78284541db832e4780e76158af&fields=id HTTP/1.1" status: 200 len: 302 time: 0.0811415 May 20 04:24:25.042446 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-786c7605-96a8-4b68-9576-762932e6f555 req-e6606f97-0454-4a6e-884f-b589b5c63010 service neutron] 10.222.0.39 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1368093 May 20 04:24:25.311641 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-786c7605-96a8-4b68-9576-762932e6f555 req-217cf0b4-a541-4f3c-acf0-f9c825567ee9 service neutron] 10.222.0.39 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2626758 May 20 04:24:25.329284 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2bd5a0f5-7250-40c9-a6ef-53c3d37c3b28 req-b932c434-c4ef-4b26-9a63-3d83abf23650 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:25.330874 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2bd5a0f5-7250-40c9-a6ef-53c3d37c3b28 req-b932c434-c4ef-4b26-9a63-3d83abf23650 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.1184576 May 20 04:24:25.374600 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2bd5a0f5-7250-40c9-a6ef-53c3d37c3b28 req-759589d3-0e80-4f8a-a237-53e88ab9a852 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0376577 May 20 04:24:25.492437 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-76560ebb-3b17-47fa-a2de-76b46be45f70 req-8438cac0-c2b6-46eb-a3a0-bed036e96514 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 210 time: 0.0451002 May 20 04:24:25.952872 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 36816) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:26.013913 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_v2 [None req-5565ad78-72b2-4670-8f64-475aa52aa085 admin admin] Deleting subnet b9f2dbbb-b724-44b9-878b-2267c4d95aee {{(pid=91643) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 20 04:24:26.017220 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5565ad78-72b2-4670-8f64-475aa52aa085 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:26.065413 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5565ad78-72b2-4670-8f64-475aa52aa085 admin admin] Notify callbacks [] for subnet, precommit_delete_associations {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:26.072589 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5565ad78-72b2-4670-8f64-475aa52aa085 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-7786687'] for subnet, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:26.319756 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5565ad78-72b2-4670-8f64-475aa52aa085 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-7786632', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868'] for subnet, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:26.324500 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:26.357943 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5565ad78-72b2-4670-8f64-475aa52aa085 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=b9f2dbbb-b724-44b9-878b-2267c4d95aee,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:26.360607 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-5565ad78-72b2-4670-8f64-475aa52aa085 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.036s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:26.363521 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5565ad78-72b2-4670-8f64-475aa52aa085 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:26.369952 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-5565ad78-72b2-4670-8f64-475aa52aa085 admin admin] 10.222.0.39 "DELETE /v2.0/subnets/b9f2dbbb-b724-44b9-878b-2267c4d95aee HTTP/1.1" status: 204 len: 168 time: 0.4159942 May 20 04:24:26.372598 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 36826) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:26.529900 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-593607'] for network, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:26.735790 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5fd7c638-599d-4978-bbf9-fd4dd647e94f req-b787f79f-7960-407a-add7-5289d53339bb tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 210 time: 0.0961421 May 20 04:24:26.855101 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-7789108', 'neutron.services.segments.db._delete_segments_for_network-8760688958082'] for network, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:26.873279 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-786c7605-96a8-4b68-9576-762932e6f555 req-c18b5c55-e740-42f0-a5d9-b29fa23acfe5 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:26.874666 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-786c7605-96a8-4b68-9576-762932e6f555 req-c18b5c55-e740-42f0-a5d9-b29fa23acfe5 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.74 "GET /v2.0/networks?id=54fc8d38-193b-49c4-b873-c5353723791f HTTP/1.1" status: 200 len: 815 time: 0.3425527 May 20 04:24:26.883881 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1b7dbd2d-12fc-40ab-89ae-e359096d85c8 req-f85bc8fb-4bbe-43e4-9d49-3f0a86b89410 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:26.885263 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1b7dbd2d-12fc-40ab-89ae-e359096d85c8 req-f85bc8fb-4bbe-43e4-9d49-3f0a86b89410 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.3943377 May 20 04:24:26.888326 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Subscribe: > segment before_delete 55550000 {{(pid=91643) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:24:26.888664 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Subscribe: > subnet precommit_delete_associations 55550000 {{(pid=91643) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:24:26.889383 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Subscribe: > segment_host_mapping after_create 55550000 {{(pid=91643) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:24:26.889722 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Subscribe: > port after_create 55550000 {{(pid=91643) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:24:26.890067 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Subscribe: > port after_delete 55550000 {{(pid=91643) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:24:26.890389 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Subscribe: > port after_update 55550000 {{(pid=91643) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:24:26.890699 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Subscribe: > segment after_delete 55550000 {{(pid=91643) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:24:26.891005 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Subscribe: > subnet after_create 55550000 {{(pid=91643) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:24:26.891344 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Subscribe: > subnet after_delete 55550000 {{(pid=91643) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:24:26.891652 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Subscribe: > subnet after_update 55550000 {{(pid=91643) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:24:26.968827 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Subscribe: > subnet after_create 55550000 {{(pid=91643) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:24:26.969191 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Subscribe: > subnet after_delete 55550000 {{(pid=91643) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:24:26.969529 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Subscribe: > subnet before_create 55550000 {{(pid=91643) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:24:26.969850 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Subscribe: > subnet before_update 55550000 {{(pid=91643) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 20 04:24:26.970212 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['a009b619-8ab3-40e3-b5f3-9cc97ccd3415']}} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:26.984810 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [req-786c7605-96a8-4b68-9576-762932e6f555 req-8032a9e3-ff70-434d-a3f1-275d35ca238c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Request body: {'port': {'device_id': 'ad654613-8b28-495c-9d73-8e92df6e2464', 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f', 'admin_state_up': True, 'tenant_id': '51fcbc78284541db832e4780e76158af'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:27.037696 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '3c4a2238-8a32-4d92-a1b1-8af91c1c27a7') {'for_net_delete': True} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:27.037998 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '3c4a2238-8a32-4d92-a1b1-8af91c1c27a7') {} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:27.103092 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8760688968331', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-1472741'] for segment, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:27.141122 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:27.157025 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1b7dbd2d-12fc-40ab-89ae-e359096d85c8 req-1ba4a2e4-3348-4bbe-9027-5fa1a77af2e5 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1591961 May 20 04:24:27.158076 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Releasing vlan 814 on physical network physnet1 to pool {{(pid=91643) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 20 04:24:27.158452 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2070625'] for segment, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: from cryptography.utils import int_from_bytes May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: from cryptography.utils import int_from_bytes May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: Traceback (most recent call last): May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: timer() May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: cb(*args, **kw) May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: return func(*args, **kwargs) May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: self._notify() May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: self.callback(batched_events) May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: event.method(event) May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: aggregate_id = self._get_aggregate_id(event.segment_id) May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: aggregate_uuid = self.p_client.list_aggregates( May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: return f(self, *a, **k) May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: return self._get(url).json() May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: return self._client.get(url, endpoint_filter=self._ks_filter, May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: return self.request(url, 'GET', **kwargs) May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: base_url = self.get_endpoint(auth, allow=allow, May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: auth = self._auth_required(auth, 'determine endpoint URL') May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 20 04:24:27.176301 dv-cinder-782405-9-iscsi neutron-server[91643]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 20 04:24:27.266437 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [req-786c7605-96a8-4b68-9576-762932e6f555 req-8032a9e3-ff70-434d-a3f1-275d35ca238c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:27.372205 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-7789117', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-8053321'] for network, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:27.380801 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [req-786c7605-96a8-4b68-9576-762932e6f555 req-8032a9e3-ff70-434d-a3f1-275d35ca238c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:27.395460 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:27.407795 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:27.415096 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-786c7605-96a8-4b68-9576-762932e6f555 req-8032a9e3-ff70-434d-a3f1-275d35ca238c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Made reservation on behalf of 51fcbc78284541db832e4780e76158af for: {'port': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:24:27.418474 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-786c7605-96a8-4b68-9576-762932e6f555 req-8032a9e3-ff70-434d-a3f1-275d35ca238c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:27.443189 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Pushing event deleted for resources: {'Network': ['ID=a009b619-8ab3-40e3-b5f3-9cc97ccd3415,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:27.449785 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:27.454520 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] 10.222.0.39 "DELETE /v2.0/networks/a009b619-8ab3-40e3-b5f3-9cc97ccd3415 HTTP/1.1" status: 204 len: 168 time: 1.0809944 May 20 04:24:27.546199 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8a010512-2f49-4682-a854-14af85e910b4 req-96a24084-5c25-4305-b866-9597ad6d5283 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:27.547678 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8a010512-2f49-4682-a854-14af85e910b4 req-96a24084-5c25-4305-b866-9597ad6d5283 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.2789114 May 20 04:24:27.638548 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 36854) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:27.647841 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8a010512-2f49-4682-a854-14af85e910b4 req-d14eb9c8-3022-4f11-9162-73702f22f5d9 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0663705 May 20 04:24:27.652142 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [req-786c7605-96a8-4b68-9576-762932e6f555 req-8032a9e3-ff70-434d-a3f1-275d35ca238c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Allocated IP 172.20.0.9 (54fc8d38-193b-49c4-b873-c5353723791f/0f918085-8d33-4e36-a59e-7f270a2b1874/b6bee055-19d1-4e07-bd15-16290eac96d9) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:24:27.694555 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-9500983d-5146-4579-b5dc-63ed32d4682b admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=0b29d684ade544028fa17828cde46d56&name=default HTTP/1.1" status: 200 len: 2975 time: 0.0549786 May 20 04:24:27.697001 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 36860) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:27.758627 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-50771e98-6b94-4ef3-a415-888b2767ab9e admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:27.785435 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-50771e98-6b94-4ef3-a415-888b2767ab9e admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:27.842810 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-50771e98-6b94-4ef3-a415-888b2767ab9e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:27.848136 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-c4486dbf-3e24-455a-aebe-939887fe0784 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:27.864334 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-50771e98-6b94-4ef3-a415-888b2767ab9e admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=06880d35-d748-4b5a-88d1-1192a88678c7,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:27.867045 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-50771e98-6b94-4ef3-a415-888b2767ab9e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:27.871498 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-786c7605-96a8-4b68-9576-762932e6f555 req-8032a9e3-ff70-434d-a3f1-275d35ca238c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:27.884582 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-50771e98-6b94-4ef3-a415-888b2767ab9e admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:27.885985 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-50771e98-6b94-4ef3-a415-888b2767ab9e admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/06880d35-d748-4b5a-88d1-1192a88678c7 HTTP/1.1" status: 204 len: 168 time: 0.1881084 May 20 04:24:27.941478 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b440cd7e-4c23-4d94-b86e-0fe191868b42 req-c588770d-68f5-44b8-9bdb-e1170c4f63d9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 210 time: 0.0464499 May 20 04:24:27.952466 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-786c7605-96a8-4b68-9576-762932e6f555 req-8032a9e3-ff70-434d-a3f1-275d35ca238c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:27.958994 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:27.967238 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['849868be-af51-4967-9a6c-93d23cecb450', '643e5696-1a48-46bc-9a75-8b39f176b597'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:24:28.034827 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-786c7605-96a8-4b68-9576-762932e6f555 req-8032a9e3-ff70-434d-a3f1-275d35ca238c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Network 54fc8d38-193b-49c4-b873-c5353723791f is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:24:28.037366 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [req-786c7605-96a8-4b68-9576-762932e6f555 req-8032a9e3-ff70-434d-a3f1-275d35ca238c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:24:28.041578 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [req-786c7605-96a8-4b68-9576-762932e6f555 req-8032a9e3-ff70-434d-a3f1-275d35ca238c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:24:28.072114 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8032a9e3-ff70-434d-a3f1-275d35ca238c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Pushing event updated for resources: {'Port': ['ID=b6bee055-19d1-4e07-bd15-16290eac96d9,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:28.079162 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-8032a9e3-ff70-434d-a3f1-275d35ca238c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.120s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:28.117186 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-786c7605-96a8-4b68-9576-762932e6f555 req-8032a9e3-ff70-434d-a3f1-275d35ca238c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:28.118074 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-786c7605-96a8-4b68-9576-762932e6f555 req-8032a9e3-ff70-434d-a3f1-275d35ca238c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:28.149650 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-786c7605-96a8-4b68-9576-762932e6f555 req-8032a9e3-ff70-434d-a3f1-275d35ca238c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.2633400 May 20 04:24:28.162145 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Request body: {'port': {'device_id': 'ad654613-8b28-495c-9d73-8e92df6e2464', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:28.326660 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:28.675617 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id b6bee055-19d1-4e07-bd15-16290eac96d9 {{(pid=91643) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:24:28.684166 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:28.702876 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.provisioning_blocks [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Transition to ACTIVE for port object b6bee055-19d1-4e07-bd15-16290eac96d9 will not be triggered until provisioned by entity L2. {{(pid=91643) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:24:28.707627 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:28.713362 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:28.797320 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 36900) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:28.867615 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Pushing event updated for resources: {'Port': ['ID=b6bee055-19d1-4e07-bd15-16290eac96d9,revision_number=2']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:28.877558 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.164s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:28.919033 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Request body: {'network': {'name': 'tempest-AttachVolumeTestJSON-97673996-network', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:28.924202 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:28.928058 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Usage tracker for resource:network and tenant:198bc8ab083f4a729f32b8ef27a6ee77 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:28.930837 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:28.936977 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:28.943751 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Attempting to bind port b6bee055-19d1-4e07-bd15-16290eac96d9 on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:24:28.944095 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Attempting to bind port b6bee055-19d1-4e07-bd15-16290eac96d9 by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': '6fe99325-a72c-48bf-abc7-9bbcc4b753ff', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 836, 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f'}] {{(pid=91643) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:24:28.944546 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Attempting to bind port b6bee055-19d1-4e07-bd15-16290eac96d9 on network 54fc8d38-193b-49c4-b873-c5353723791f {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:24:28.946888 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Made reservation on behalf of 198bc8ab083f4a729f32b8ef27a6ee77 for: {'network': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:24:28.949330 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-585852'] for network, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:28.949467 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Port b6bee055-19d1-4e07-bd15-16290eac96d9 on network 54fc8d38-193b-49c4-b873-c5353723791f not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:24:28.949713 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Attempting to bind port b6bee055-19d1-4e07-bd15-16290eac96d9 on network 54fc8d38-193b-49c4-b873-c5353723791f {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:24:28.954687 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 24, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:24:28.955048 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Checking segment: {'id': '6fe99325-a72c-48bf-abc7-9bbcc4b753ff', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 836, 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91643) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:24:28.955456 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Bound using segment: {'id': '6fe99325-a72c-48bf-abc7-9bbcc4b753ff', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 836, 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f'} {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:24:28.955878 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Bound port: b6bee055-19d1-4e07-bd15-16290eac96d9, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '6fe99325-a72c-48bf-abc7-9bbcc4b753ff', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 836, 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f'}}] {{(pid=91643) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:24:28.956213 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:28.961388 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:28.976772 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:28.980503 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Usage tracker for resource:security_group_rule and tenant:198bc8ab083f4a729f32b8ef27a6ee77 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:28.983325 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:28.989114 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:29.051467 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] For port b6bee055-19d1-4e07-bd15-16290eac96d9, host hv-cinder-78240, cleared binding levels {{(pid=91643) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:24:29.055408 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] For port b6bee055-19d1-4e07-bd15-16290eac96d9, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b6bee055-19d1-4e07-bd15-16290eac96d9,segment=NetworkSegment(6fe99325-a72c-48bf-abc7-9bbcc4b753ff),segment_id=6fe99325-a72c-48bf-abc7-9bbcc4b753ff)] {{(pid=91643) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:24:29.083530 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:29.087748 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:29.092390 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-a35e4b54-eabe-4bc7-98c0-9d88b420d72b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:29.096524 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:29.110327 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:29.117002 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 808} {{(pid=91644) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 20 04:24:29.132813 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f17ceb35-5276-4876-9705-17e2c336a41c,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:29.136468 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:29.139586 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:29.140084 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.db.segments_db [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Added segment b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7 of type vlan for network 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d May 20 04:24:29.163235 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:29.163839 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:29.165216 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-786c7605-96a8-4b68-9576-762932e6f555 req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] 10.222.0.74 "PUT /v2.0/ports/b6bee055-19d1-4e07-bd15-16290eac96d9 HTTP/1.1" status: 200 len: 1093 time: 1.0079260 May 20 04:24:29.165675 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'ad654613-8b28-495c-9d73-8e92df6e2464', 'tag': 'b6bee055-19d1-4e07-bd15-16290eac96d9'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:24:29.173337 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8760688516720'] for network, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:29.208553 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915'] for network, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:29.210116 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:29.210742 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:29.214890 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:29.316031 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Pushing event updated for resources: {'Network': ['ID=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:29.325806 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.111s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:29.327430 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Network 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d details requested by agent ovs-agent-dv-cinder-782405-9-iscsi with host dv-cinder-782405-9-iscsi {{(pid=91645) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 20 04:24:29.328757 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] 10.222.0.39 "POST /v2.0/networks HTTP/1.1" status: 201 len: 882 time: 0.5303588 May 20 04:24:29.334802 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 36914) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:29.336918 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'ad654613-8b28-495c-9d73-8e92df6e2464', 'tag': 'b6bee055-19d1-4e07-bd15-16290eac96d9', 'status': 'completed', 'code': 200} May 20 04:24:29.341111 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-c0e8eb3d-1465-402a-a2b4-5c18e70102e0 admin admin] Request body: {'subnet': {'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'cidr': '172.20.0.0/28', 'name': 'tempest-AttachVolumeTestJSON-97673996-subnet', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'enable_dhcp': True, 'ip_version': 4}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:29.346364 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-c0e8eb3d-1465-402a-a2b4-5c18e70102e0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:29.351185 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-c0e8eb3d-1465-402a-a2b4-5c18e70102e0 admin admin] Usage tracker for resource:subnet and tenant:198bc8ab083f4a729f32b8ef27a6ee77 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:29.353322 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5e8c483-ab99-4601-a7f5-5de5f44b6697 req-8bfda303-9165-4834-abd2-951220eac25c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:29.354038 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-b5e8c483-ab99-4601-a7f5-5de5f44b6697 req-8bfda303-9165-4834-abd2-951220eac25c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Synchronizing usage tracker for tenant:51fcbc78284541db832e4780e76158af on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:24:29.354506 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-c0e8eb3d-1465-402a-a2b4-5c18e70102e0 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:29.359345 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Pushing event updated for resources: {'Port': ['ID=b6bee055-19d1-4e07-bd15-16290eac96d9,revision_number=3']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:29.369186 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-c0e8eb3d-1465-402a-a2b4-5c18e70102e0 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:29.374634 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.264s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:29.383758 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-c0e8eb3d-1465-402a-a2b4-5c18e70102e0 admin admin] Made reservation on behalf of 198bc8ab083f4a729f32b8ef27a6ee77 for: {'subnet': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:24:29.387711 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c0e8eb3d-1465-402a-a2b4-5c18e70102e0 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-1810190'] for subnet, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:29.391495 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-b5e8c483-ab99-4601-a7f5-5de5f44b6697 req-8bfda303-9165-4834-abd2-951220eac25c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Unset dirty status for tenant:51fcbc78284541db832e4780e76158af on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:24:29.393117 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b5e8c483-ab99-4601-a7f5-5de5f44b6697 req-8bfda303-9165-4834-abd2-951220eac25c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 943 time: 0.3363597 May 20 04:24:29.455308 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-786c7605-96a8-4b68-9576-762932e6f555 req-60581077-80a7-43bd-8b9a-bb39e9fb3c8a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:29.456993 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-786c7605-96a8-4b68-9576-762932e6f555 req-60581077-80a7-43bd-8b9a-bb39e9fb3c8a service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=51fcbc78284541db832e4780e76158af&device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 1096 time: 0.2443190 May 20 04:24:29.469057 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b5e8c483-ab99-4601-a7f5-5de5f44b6697 req-0b698b01-d088-41f8-a905-1b5b8f10b013 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0667896 May 20 04:24:29.536217 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-786c7605-96a8-4b68-9576-762932e6f555 req-a1965320-5229-4fb8-a326-f201dfb99db5 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=b6bee055-19d1-4e07-bd15-16290eac96d9 HTTP/1.1" status: 200 len: 217 time: 0.0707450 May 20 04:24:29.601571 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-786c7605-96a8-4b68-9576-762932e6f555 req-e6f472e8-f561-4313-808b-b7309243d37b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:29.602940 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-786c7605-96a8-4b68-9576-762932e6f555 req-e6f472e8-f561-4313-808b-b7309243d37b service neutron] 10.222.0.74 "GET /v2.0/subnets?id=0f918085-8d33-4e36-a59e-7f270a2b1874 HTTP/1.1" status: 200 len: 855 time: 0.0588086 May 20 04:24:29.650550 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-786c7605-96a8-4b68-9576-762932e6f555 req-df813a70-f929-4b87-863f-8c225f1c1f6e service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=54fc8d38-193b-49c4-b873-c5353723791f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0408599 May 20 04:24:29.791621 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-786c7605-96a8-4b68-9576-762932e6f555 req-ec8bfedc-d687-4ccb-a1c0-79de1e6cc592 service neutron] 10.222.0.74 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1237130 May 20 04:24:29.809887 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c0e8eb3d-1465-402a-a2b4-5c18e70102e0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-1988978', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-1810236'] for subnet, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:29.814446 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:29.889858 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c0e8eb3d-1465-402a-a2b4-5c18e70102e0 admin admin] Pushing event updated for resources: {'Subnet': ['ID=4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3,revision_number=0']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:29.895004 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-c0e8eb3d-1465-402a-a2b4-5c18e70102e0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.080s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:29.901460 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c0e8eb3d-1465-402a-a2b4-5c18e70102e0 admin admin] Checking segment: {'id': 'b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 808, 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:24:29.901813 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c0e8eb3d-1465-402a-a2b4-5c18e70102e0 admin admin] Network hostable DHCP agents. Network: 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d, hostable agents: ['fffae25a-f450-4182-8f9e-ac29d143f6c9'], hosted agents: [] {{(pid=91644) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 20 04:24:29.920758 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c0e8eb3d-1465-402a-a2b4-5c18e70102e0 admin admin] Network 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d is scheduled to be hosted by DHCP agent fffae25a-f450-4182-8f9e-ac29d143f6c9 with binding_index 1 {{(pid=91644) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 20 04:24:29.926436 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-786c7605-96a8-4b68-9576-762932e6f555 req-89ccf287-4615-4272-bf4f-7f18d4342a6b service neutron] 10.222.0.74 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1279583 May 20 04:24:29.931249 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-c0e8eb3d-1465-402a-a2b4-5c18e70102e0 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-782405-9-iscsi, 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d) {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:24:29.940287 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-c0e8eb3d-1465-402a-a2b4-5c18e70102e0 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:29.940869 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c0e8eb3d-1465-402a-a2b4-5c18e70102e0 admin admin] Notify callbacks [] for subnet, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:29.942694 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-76a4648f-bb30-439b-a257-5063946f9f6d None None] Network 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:24:29.972271 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 36924) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:29.973597 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-c0e8eb3d-1465-402a-a2b4-5c18e70102e0 admin admin] 10.222.0.39 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 845 time: 0.6375163 May 20 04:24:29.977592 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Request body: {'router': {'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:29.983004 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:30.026090 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Usage tracker for resource:router and tenant:198bc8ab083f4a729f32b8ef27a6ee77 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:30.033535 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:30.041231 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:30.066164 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-8071540', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-260712'] for router, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:30.067570 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-8056436', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-8053301', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-7881139', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-260707'] for router, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:30.095776 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-18f3822f-126b-4a25-97e4-3f5fd028a59a req-f9a0a12b-4ccd-4d31-99a9-14916430e5fc service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:30.097262 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-18f3822f-126b-4a25-97e4-3f5fd028a59a req-f9a0a12b-4ccd-4d31-99a9-14916430e5fc service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=51fcbc78284541db832e4780e76158af&device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 1096 time: 0.1154363 May 20 04:24:30.125586 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:30.177226 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcp85c0b5c8-c799-5455-967d-3e9c2c8b4839-49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'fixed_ips': [{'subnet_id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3'}]}} from dv-cinder-782405-9-iscsi. {{(pid=91645) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 20 04:24:30.178291 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:30.184217 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 36930) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:30.307617 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b74cab4e-b81c-4278-8976-ef7657eef473 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:30.321856 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-b74cab4e-b81c-4278-8976-ef7657eef473 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:30.325623 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-b74cab4e-b81c-4278-8976-ef7657eef473 admin admin] Usage tracker for resource:security_group_rule and tenant:7edbf5be74f549fdb7e53bcd3bc00037 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:30.328424 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-b74cab4e-b81c-4278-8976-ef7657eef473 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:30.332321 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.db_base_plugin_common [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Allocated IP 172.20.0.2 (49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d/4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3/4f7409b5-c762-4162-ae2d-d2066230c4f4) {{(pid=91645) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:24:30.335363 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-b74cab4e-b81c-4278-8976-ef7657eef473 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:30.424378 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-18f3822f-126b-4a25-97e4-3f5fd028a59a req-8952a130-ed30-4790-a247-3e4c288cf0aa service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:30.425693 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-18f3822f-126b-4a25-97e4-3f5fd028a59a req-8952a130-ed30-4790-a247-3e4c288cf0aa service neutron] 10.222.0.74 "GET /v2.0/networks?id=54fc8d38-193b-49c4-b873-c5353723791f HTTP/1.1" status: 200 len: 916 time: 0.3213773 May 20 04:24:30.433406 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b74cab4e-b81c-4278-8976-ef7657eef473 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:30.439793 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-b74cab4e-b81c-4278-8976-ef7657eef473 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:30.449217 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-1e5ac10d-e8d3-4ecb-be30-1db7c1f2857d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:30.510533 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-b74cab4e-b81c-4278-8976-ef7657eef473 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=7edbf5be74f549fdb7e53bcd3bc00037&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3252263 May 20 04:24:30.514067 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 36944) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:30.529161 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b74cab4e-b81c-4278-8976-ef7657eef473 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=4e17d81b-1046-48b4-a17d-dbe574cb130b,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:30.556818 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-b74cab4e-b81c-4278-8976-ef7657eef473 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.108s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:30.570866 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Notify callbacks [] for port, precommit_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:30.581446 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Transition to ACTIVE for port object 4f7409b5-c762-4162-ae2d-d2066230c4f4 will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:24:30.621410 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Transition to ACTIVE for port object 4f7409b5-c762-4162-ae2d-d2066230c4f4 will not be triggered until provisioned by entity DHCP. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:24:30.622004 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-fbc095e1-e940-4daf-bfcc-10e600e5a9e8 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:30.633270 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:30.642399 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-8053308', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-8071430', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-260685'] for router, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:30.645171 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:30.652217 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-fbc095e1-e940-4daf-bfcc-10e600e5a9e8 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:30.688306 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-18f3822f-126b-4a25-97e4-3f5fd028a59a req-05d30c40-631f-48d3-b076-74502f1d143e service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=b6bee055-19d1-4e07-bd15-16290eac96d9 HTTP/1.1" status: 200 len: 217 time: 0.1510093 May 20 04:24:30.714563 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-fbc095e1-e940-4daf-bfcc-10e600e5a9e8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:30.719689 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-84f06910-0f20-4ff3-9726-517986e2fe73 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:30.727493 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Network 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d is already hosted by enough agents. {{(pid=91645) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:24:30.731587 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Pushing event updated for resources: {'Port': ['ID=4f7409b5-c762-4162-ae2d-d2066230c4f4,revision_number=1']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:30.732436 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fbc095e1-e940-4daf-bfcc-10e600e5a9e8 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=4e17d81b-1046-48b4-a17d-dbe574cb130b,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:30.734796 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.base_scheduler [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Resources bound (agent type, host, resource id): {{(pid=91645) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:24:30.737193 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.092s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:30.738487 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Received port after_create {{(pid=91645) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:24:30.744836 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-fbc095e1-e940-4daf-bfcc-10e600e5a9e8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.025s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:30.750605 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-fbc095e1-e940-4daf-bfcc-10e600e5a9e8 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:30.751807 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-fbc095e1-e940-4daf-bfcc-10e600e5a9e8 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/4e17d81b-1046-48b4-a17d-dbe574cb130b HTTP/1.1" status: 204 len: 168 time: 0.2368400 May 20 04:24:30.800563 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:24:30.835293 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-18f3822f-126b-4a25-97e4-3f5fd028a59a req-cc0f1771-d100-462c-81a5-d7a77088e547 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:30.836833 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-18f3822f-126b-4a25-97e4-3f5fd028a59a req-cc0f1771-d100-462c-81a5-d7a77088e547 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=0f918085-8d33-4e36-a59e-7f270a2b1874 HTTP/1.1" status: 200 len: 855 time: 0.1409101 May 20 04:24:30.883939 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Attempting to bind port 4f7409b5-c762-4162-ae2d-d2066230c4f4 on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:24:30.884345 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Attempting to bind port 4f7409b5-c762-4162-ae2d-d2066230c4f4 by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': 'b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 808, 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:24:30.884767 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Attempting to bind port 4f7409b5-c762-4162-ae2d-d2066230c4f4 on network 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:24:30.891336 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 24, 15), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:24:30.891921 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Checking segment: {'id': 'b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 808, 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:24:30.892560 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:30.892959 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Bound using segment: {'id': 'b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 808, 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:24:30.893528 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Bound port: 4f7409b5-c762-4162-ae2d-d2066230c4f4, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 808, 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:24:30.893864 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:30.974099 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e6c361cc-4f46-4439-87e0-6f6e2f1ce713 req-fca9483c-3791-4538-827d-3f2853d6e113 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:30.975658 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e6c361cc-4f46-4439-87e0-6f6e2f1ce713 req-fca9483c-3791-4538-827d-3f2853d6e113 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=bec56d8703cf47f3b7193b7d1f55396d&device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 1098 time: 0.0995035 May 20 04:24:31.022101 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] For port 4f7409b5-c762-4162-ae2d-d2066230c4f4, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:24:31.030626 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] For port 4f7409b5-c762-4162-ae2d-d2066230c4f4, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=4f7409b5-c762-4162-ae2d-d2066230c4f4,segment=NetworkSegment(b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7),segment_id=b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:24:31.036072 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:31.045944 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:31.054159 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-18f3822f-126b-4a25-97e4-3f5fd028a59a req-4f8c0bad-4d7b-405c-aa7f-476acb9150d3 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=54fc8d38-193b-49c4-b873-c5353723791f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.1953011 May 20 04:24:31.067090 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-14fb81a4-1174-444f-a769-0c3224a73237 req-a4f576b4-e54a-4c77-ba6c-3f52014b560d tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:31.068798 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-14fb81a4-1174-444f-a769-0c3224a73237 req-a4f576b4-e54a-4c77-ba6c-3f52014b560d tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 943 time: 0.4287827 May 20 04:24:31.150040 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Pushing event updated for resources: {'Port': ['ID=4f7409b5-c762-4162-ae2d-d2066230c4f4,revision_number=2']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:31.150855 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e6c361cc-4f46-4439-87e0-6f6e2f1ce713 req-61054dde-d983-45e0-8520-3e96e4e41f77 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:31.152855 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e6c361cc-4f46-4439-87e0-6f6e2f1ce713 req-61054dde-d983-45e0-8520-3e96e4e41f77 service neutron] 10.222.0.74 "GET /v2.0/networks?id=36b37213-8439-447d-9eba-a78baf7c698e HTTP/1.1" status: 200 len: 920 time: 0.1697674 May 20 04:24:31.153494 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.108s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:31.247238 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e6c361cc-4f46-4439-87e0-6f6e2f1ce713 req-35029ada-6d59-49d6-9278-4c240083ff41 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=706277c6-cd8c-4146-ab91-ed9504446690 HTTP/1.1" status: 200 len: 217 time: 0.0847092 May 20 04:24:31.267854 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-14fb81a4-1174-444f-a769-0c3224a73237 req-85c84072-fb41-4de0-955f-bbefcab79dfb tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1761334 May 20 04:24:31.350329 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-18f3822f-126b-4a25-97e4-3f5fd028a59a req-38c59824-50f1-441d-a9b8-7217c5b3e3c1 service neutron] 10.222.0.74 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2803981 May 20 04:24:31.442870 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e6c361cc-4f46-4439-87e0-6f6e2f1ce713 req-d0592c78-4883-4467-94ed-4f1240a3adeb service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:31.444357 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e6c361cc-4f46-4439-87e0-6f6e2f1ce713 req-d0592c78-4883-4467-94ed-4f1240a3adeb service neutron] 10.222.0.74 "GET /v2.0/subnets?id=b25ae9c8-e11d-45ee-bde2-332cc1538602 HTTP/1.1" status: 200 len: 859 time: 0.1858277 May 20 04:24:31.511470 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e6c361cc-4f46-4439-87e0-6f6e2f1ce713 req-0815b9de-8032-4262-a78d-4c5dfbac20db service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=36b37213-8439-447d-9eba-a78baf7c698e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0551722 May 20 04:24:31.667205 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e6c361cc-4f46-4439-87e0-6f6e2f1ce713 req-42f08e02-5fd5-4e47-b524-99d81069ed03 service neutron] 10.222.0.74 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1482677 May 20 04:24:31.703123 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 36974) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:31.730547 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:31.768963 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:31.772110 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-18f3822f-126b-4a25-97e4-3f5fd028a59a req-3de751fa-4335-4df2-8560-aad5f869c3a2 service neutron] 10.222.0.74 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3268309 May 20 04:24:31.780681 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a65f947a-f14f-48a9-87af-9c01e141f0a0 req-60b00887-868a-477b-bc1b-7bc774dfd8e3 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:31.782133 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a65f947a-f14f-48a9-87af-9c01e141f0a0 req-60b00887-868a-477b-bc1b-7bc774dfd8e3 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.4868557 May 20 04:24:31.829016 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e6c361cc-4f46-4439-87e0-6f6e2f1ce713 req-61136423-f1da-4668-bb79-a798fb401a27 service neutron] 10.222.0.74 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1522422 May 20 04:24:31.838058 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Request body: {'network': {'name': 'tempest-TestVolumeBootPattern-59002837-network', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:31.843157 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:31.847408 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Usage tracker for resource:network and tenant:0e3baa0ea24d42f3ab59e40845b350fd is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:31.850411 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:31.856193 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:31.865591 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Made reservation on behalf of 0e3baa0ea24d42f3ab59e40845b350fd for: {'network': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:24:31.868009 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-585852'] for network, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:31.880113 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:31.887492 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a65f947a-f14f-48a9-87af-9c01e141f0a0 req-b40d4426-6ebe-4290-9ec6-636c50e9be11 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0695274 May 20 04:24:31.895122 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:31.899193 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Usage tracker for resource:security_group_rule and tenant:0e3baa0ea24d42f3ab59e40845b350fd is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:31.908155 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:31.916058 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:31.968707 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Allocated IP 172.27.0.33 (712ce8c4-8feb-4719-bf13-e06992125cd9/e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5/c3a0d63d-eb15-4b80-a8a4-15678c28b534) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:24:32.014231 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:32.020286 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:32.025640 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-99a62b74-28b7-4461-9ff4-c9328c07b4c2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:32.050363 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 841} {{(pid=91644) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 20 04:24:32.068219 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=9f4aef68-406c-46a5-9e8f-df01ef8e502f,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:32.073397 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:32.074826 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:32.075161 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.db.segments_db [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Added segment 05406cc0-328f-4fba-b043-20b2b57b274d of type vlan for network 14c537db-01aa-4360-a348-47f94c75e03b May 20 04:24:32.106800 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8760688516720'] for network, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:32.143962 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915'] for network, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:32.145568 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:32.146148 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:32.149871 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-1b418a50-34b5-4a12-88c5-a085e44167f3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:32.161064 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:32.198451 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:32.202854 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:32.253178 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Pushing event updated for resources: {'Network': ['ID=14c537db-01aa-4360-a348-47f94c75e03b,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:32.260415 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] 10.222.0.39 "POST /v2.0/networks HTTP/1.1" status: 201 len: 883 time: 0.5523999 May 20 04:24:32.260415 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 36986) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:32.260979 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.111s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:32.266420 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-11509166-f448-483d-87f2-6a36601237da admin admin] Request body: {'subnet': {'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestVolumeBootPattern-59002837-subnet', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'ip_version': 4}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:32.269347 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Network 14c537db-01aa-4360-a348-47f94c75e03b details requested by agent ovs-agent-dv-cinder-782405-9-iscsi with host dv-cinder-782405-9-iscsi {{(pid=91645) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 20 04:24:32.271290 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-11509166-f448-483d-87f2-6a36601237da admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:32.274834 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:24:32.274967 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-11509166-f448-483d-87f2-6a36601237da admin admin] Usage tracker for resource:subnet and tenant:0e3baa0ea24d42f3ab59e40845b350fd is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:32.277762 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-11509166-f448-483d-87f2-6a36601237da admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:32.279686 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:24:32.283623 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-11509166-f448-483d-87f2-6a36601237da admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:32.285634 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:24:32.287780 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-bc6a61d9-79a4-4ce8-9848-d75b9c9ed3e3 None None] Network 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:24:32.295037 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-11509166-f448-483d-87f2-6a36601237da admin admin] Made reservation on behalf of 0e3baa0ea24d42f3ab59e40845b350fd for: {'subnet': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:24:32.306761 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-11509166-f448-483d-87f2-6a36601237da admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-1810190'] for subnet, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:32.314966 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Pushing event updated for resources: {'Port': ['ID=c3a0d63d-eb15-4b80-a8a4-15678c28b534,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:32.322609 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.120s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:32.486168 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-5e353e52-8b10-42ef-a22b-3a24f8ffb342 None None] Provisioning for port 4f7409b5-c762-4162-ae2d-d2066230c4f4 completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:24:32.670548 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e45199d3-a529-46eb-a7b7-df5ed82e18ad req-bdc398cb-b6cc-45bc-bff2-fc2ba6cc9024 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:32.672095 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e45199d3-a529-46eb-a7b7-df5ed82e18ad req-bdc398cb-b6cc-45bc-bff2-fc2ba6cc9024 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 943 time: 0.2618165 May 20 04:24:32.722623 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-11509166-f448-483d-87f2-6a36601237da admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-1988978', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-1810236'] for subnet, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:32.728247 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-c0e8eb3d-1465-402a-a2b4-5c18e70102e0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:32.787811 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-11509166-f448-483d-87f2-6a36601237da admin admin] Pushing event updated for resources: {'Subnet': ['ID=cdea475c-2929-4ae4-9271-87cc2fb9a378,revision_number=0']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:32.790952 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-11509166-f448-483d-87f2-6a36601237da admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.062s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:32.825053 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-11509166-f448-483d-87f2-6a36601237da admin admin] Checking segment: {'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:24:32.827528 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-11509166-f448-483d-87f2-6a36601237da admin admin] Network hostable DHCP agents. Network: 14c537db-01aa-4360-a348-47f94c75e03b, hostable agents: ['fffae25a-f450-4182-8f9e-ac29d143f6c9'], hosted agents: [] {{(pid=91644) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 20 04:24:32.845588 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-11509166-f448-483d-87f2-6a36601237da admin admin] Network 14c537db-01aa-4360-a348-47f94c75e03b is scheduled to be hosted by DHCP agent fffae25a-f450-4182-8f9e-ac29d143f6c9 with binding_index 1 {{(pid=91644) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 20 04:24:32.856195 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-11509166-f448-483d-87f2-6a36601237da admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-782405-9-iscsi, 14c537db-01aa-4360-a348-47f94c75e03b) {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:24:32.865473 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-3b406cd8-80f8-4cc9-93c7-b4e02837a241 None None] Network 14c537db-01aa-4360-a348-47f94c75e03b requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:24:32.865817 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-11509166-f448-483d-87f2-6a36601237da admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:32.866392 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-11509166-f448-483d-87f2-6a36601237da admin admin] Notify callbacks [] for subnet, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:32.890483 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-11509166-f448-483d-87f2-6a36601237da admin admin] 10.222.0.39 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 846 time: 0.6260488 May 20 04:24:32.890714 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 36994) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:32.891342 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e45199d3-a529-46eb-a7b7-df5ed82e18ad req-901a48f9-79d5-4d82-88fd-274bebfe4f78 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2092946 May 20 04:24:32.896509 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Request body: {'router': {'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:32.902076 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:32.906957 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Usage tracker for resource:router and tenant:0e3baa0ea24d42f3ab59e40845b350fd is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:32.910925 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:32.917001 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:32.919123 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:32.929437 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-8071540', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-260712'] for router, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:32.930680 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-8056436', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-8053301', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-7881139', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-260707'] for router, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:32.947773 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:33.165100 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcp85c0b5c8-c799-5455-967d-3e9c2c8b4839-14c537db-01aa-4360-a348-47f94c75e03b', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378'}]}} from dv-cinder-782405-9-iscsi. {{(pid=91645) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 20 04:24:33.166194 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:33.227058 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-8053308', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-8071430', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-260685'] for router, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:33.375923 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-611b0831-638b-47dc-b6d3-684c0de89f1f req-a49b3f9c-6a21-48a3-ac80-cab753cf222d tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:33.377447 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-611b0831-638b-47dc-b6d3-684c0de89f1f req-a49b3f9c-6a21-48a3-ac80-cab753cf222d tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.3171248 May 20 04:24:33.382946 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-8071551', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-8053391'] for router, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:33.385336 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.db_base_plugin_common [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Allocated IP 172.20.0.2 (14c537db-01aa-4360-a348-47f94c75e03b/cdea475c-2929-4ae4-9271-87cc2fb9a378/cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae) {{(pid=91645) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:24:33.484735 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 4f7409b5-c762-4162-ae2d-d2066230c4f4 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:24:33.504170 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-611b0831-638b-47dc-b6d3-684c0de89f1f req-5c1ec3ab-d2a8-49df-976b-9c24580cd272 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1175869 May 20 04:24:33.530058 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b6bee055-19d1-4e07-bd15-16290eac96d9 details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:24:33.620669 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Router 10aeccef-e762-4811-a411-a1c9817db442 is scheduled to L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 with binding_index 1 {{(pid=91643) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 20 04:24:33.649924 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:33.683483 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:33.778577 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b6bee055-19d1-4e07-bd15-16290eac96d9, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b6bee055-19d1-4e07-bd15-16290eac96d9,segment=NetworkSegment(6fe99325-a72c-48bf-abc7-9bbcc4b753ff),segment_id=6fe99325-a72c-48bf-abc7-9bbcc4b753ff)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:24:33.790624 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port 4f7409b5-c762-4162-ae2d-d2066230c4f4 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:24:33.800918 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': 'b6bee055-19d1-4e07-bd15-16290eac96d9', 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f', 'port_id': 'b6bee055-19d1-4e07-bd15-16290eac96d9', 'mac_address': 'fa:16:3e:e0:a3:3f', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 836, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '0f918085-8d33-4e36-a59e-7f270a2b1874', 'ip_address': '172.20.0.9'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:24:33.809073 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:24:33.810496 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 4f7409b5-c762-4162-ae2d-d2066230c4f4 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:33.810877 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:33.822878 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Sync routers for ids ['10aeccef-e762-4811-a411-a1c9817db442'] in dv-cinder-782405-9-iscsi {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:24:33.832941 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Allocated IP 172.27.0.210 (712ce8c4-8feb-4719-bf13-e06992125cd9/e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5/8f9c794c-3d7c-41d8-8927-b7ccdfe087ae) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:24:33.834668 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Notify callbacks [] for router, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:33.842737 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Notify callbacks [] for port, precommit_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:33.844309 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Routers {'10aeccef-e762-4811-a411-a1c9817db442'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:24:33.845818 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:24:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['10aeccef-e762-4811-a411-a1c9817db442'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:33.849568 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] 10.222.0.39 "POST /v2.0/routers HTTP/1.1" status: 201 len: 920 time: 3.8749394 May 20 04:24:33.851891 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 37004) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:33.910865 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Transition to ACTIVE for port object cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:24:33.935990 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:33.957210 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Transition to ACTIVE for port object cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae will not be triggered until provisioned by entity DHCP. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:24:33.972127 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id b6bee055-19d1-4e07-bd15-16290eac96d9 {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:24:33.983587 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:33.997020 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-41952050-2fb0-440f-b07b-0ebf4cd2bf12 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:34.060867 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Set router 10aeccef-e762-4811-a411-a1c9817db442 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:24:34.109613 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:34.111789 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Network 14c537db-01aa-4360-a348-47f94c75e03b is already hosted by enough agents. {{(pid=91645) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:24:34.122273 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b6bee055-19d1-4e07-bd15-16290eac96d9, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b6bee055-19d1-4e07-bd15-16290eac96d9,segment=NetworkSegment(6fe99325-a72c-48bf-abc7-9bbcc4b753ff),segment_id=6fe99325-a72c-48bf-abc7-9bbcc4b753ff)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:24:34.126124 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.base_scheduler [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Resources bound (agent type, host, resource id): {{(pid=91645) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:24:34.143996 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Received port after_create {{(pid=91645) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:24:34.146060 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:34.151387 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Pushing event updated for resources: {'Port': ['ID=cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae,revision_number=1']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:34.156578 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:34.160043 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.163s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:34.160505 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.010s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:34.171018 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:34.247133 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:24:34.250040 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:34.252115 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:24:34.259301 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:24:34.328323 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Pushing event updated for resources: {'Port': ['ID=8f9c794c-3d7c-41d8-8927-b7ccdfe087ae,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:34.333232 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.162s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:34.350291 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Attempting to bind port cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:24:34.350676 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Attempting to bind port cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:24:34.351037 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Attempting to bind port cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae on network 14c537db-01aa-4360-a348-47f94c75e03b {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:24:34.354203 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port 4f7409b5-c762-4162-ae2d-d2066230c4f4, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=4f7409b5-c762-4162-ae2d-d2066230c4f4,segment=NetworkSegment(b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7),segment_id=b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:24:34.356936 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=b6bee055-19d1-4e07-bd15-16290eac96d9,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:34.361809 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.201s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:34.363042 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 24, 15), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:24:34.363628 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Checking segment: {'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:24:34.364138 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:34.364544 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Bound using segment: {'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:24:34.365033 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Bound port: cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:24:34.365411 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:34.375420 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:34.379109 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:34.448397 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33'}], 'enable_snat': True}, 'gw_port_id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:24:32Z', 'revision_number': 3, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port': {'id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:64:1f', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_gateway', 'standard_attr_id': 295, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:31Z', 'updated_at': '2021-05-20T04:24:31Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:34.453807 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Checking router: 10aeccef-e762-4811-a411-a1c9817db442 for host: dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:24:34.454395 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Updating router 10aeccef-e762-4811-a411-a1c9817db442 port c3a0d63d-eb15-4b80-a8a4-15678c28b534 binding host dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 20 04:24:34.508473 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=4f7409b5-c762-4162-ae2d-d2066230c4f4,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:34.514131 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.135s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:34.536007 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] For port cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:24:34.540864 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] For port cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:24:34.547550 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:34.551586 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-e927cfab-32e4-429c-a060-2391d64bdb5a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:34.567828 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:34.585014 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ce0195ee-8610-4df4-a642-2432e8070780 req-f9683f81-0c40-4783-8d1a-327c5ddee18c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:34.586509 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ce0195ee-8610-4df4-a642-2432e8070780 req-f9683f81-0c40-4783-8d1a-327c5ddee18c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 944 time: 0.4757531 May 20 04:24:34.614836 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['b6bee055-19d1-4e07-bd15-16290eac96d9'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:24:34.708854 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ce0195ee-8610-4df4-a642-2432e8070780 req-d0967aad-2682-4270-b370-90ceb8c92f90 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1119859 May 20 04:24:34.780318 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Pushing event updated for resources: {'Port': ['ID=cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae,revision_number=2']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:34.799479 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.245s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:34.920337 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:35.002631 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:35.053541 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Notify callbacks [] for port, precommit_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:35.073630 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.provisioning_blocks [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Transition to ACTIVE for port object c3a0d63d-eb15-4b80-a8a4-15678c28b534 will not be triggered until provisioned by entity L2. {{(pid=91646) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:24:35.090767 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:35.098449 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:35.166148 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-97814b72-4469-4881-97dd-0c7878a51128 req-fe5115c1-e45e-4f0e-a206-9859691032cf tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:35.168647 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-97814b72-4469-4881-97dd-0c7878a51128 req-fe5115c1-e45e-4f0e-a206-9859691032cf tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.3082376 May 20 04:24:35.261196 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-97814b72-4469-4881-97dd-0c7878a51128 req-5c72d0f5-5f65-47fc-85ec-e0509923dc5b tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0845191 May 20 04:24:35.261805 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Attempting to bind port c3a0d63d-eb15-4b80-a8a4-15678c28b534 on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:24:35.262176 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Attempting to bind port c3a0d63d-eb15-4b80-a8a4-15678c28b534 by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}] {{(pid=91646) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:24:35.262591 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Attempting to bind port c3a0d63d-eb15-4b80-a8a4-15678c28b534 on network 712ce8c4-8feb-4719-bf13-e06992125cd9 {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:24:35.272559 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a8660e92-7d3f-423f-b1f3-7f578986a5b3 None None] Pushing event updated for resources: {'Port': ['ID=c3a0d63d-eb15-4b80-a8a4-15678c28b534,revision_number=2']} {{(pid=91646) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:35.282335 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8660e92-7d3f-423f-b1f3-7f578986a5b3 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:35.282667 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8660e92-7d3f-423f-b1f3-7f578986a5b3 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:35.282994 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8660e92-7d3f-423f-b1f3-7f578986a5b3 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:35.283383 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8660e92-7d3f-423f-b1f3-7f578986a5b3 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:35.283875 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8660e92-7d3f-423f-b1f3-7f578986a5b3 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:35.284198 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8660e92-7d3f-423f-b1f3-7f578986a5b3 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:35.284514 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8660e92-7d3f-423f-b1f3-7f578986a5b3 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:35.284829 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8660e92-7d3f-423f-b1f3-7f578986a5b3 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:35.285253 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8660e92-7d3f-423f-b1f3-7f578986a5b3 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:35.285527 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8660e92-7d3f-423f-b1f3-7f578986a5b3 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:35.285835 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8660e92-7d3f-423f-b1f3-7f578986a5b3 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:35.286263 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a8660e92-7d3f-423f-b1f3-7f578986a5b3 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:35.291197 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 24, 15), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:24:35.291980 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Checking segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91646) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:24:35.292478 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:35.292960 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Bound using segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:24:35.293549 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Bound port: c3a0d63d-eb15-4b80-a8a4-15678c28b534, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}}] {{(pid=91646) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:24:35.294017 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:35.328474 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-a8660e92-7d3f-423f-b1f3-7f578986a5b3 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.230s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:35.428265 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:24:35.435261 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.db [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] For port c3a0d63d-eb15-4b80-a8a4-15678c28b534, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91646) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:24:35.438633 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.db [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] For port c3a0d63d-eb15-4b80-a8a4-15678c28b534, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=c3a0d63d-eb15-4b80-a8a4-15678c28b534,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91646) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:24:35.444381 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:35.444573 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Allocated IP 172.20.0.1 (49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d/4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3/7dc69379-dd61-4add-8e0a-4ef00a211a0a) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:24:35.451004 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:35.531992 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:24:35.555791 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 4f7409b5-c762-4162-ae2d-d2066230c4f4 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:24:35.568022 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a8660e92-7d3f-423f-b1f3-7f578986a5b3 None None] Pushing event updated for resources: {'Port': ['ID=c3a0d63d-eb15-4b80-a8a4-15678c28b534,revision_number=3']} {{(pid=91646) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:35.579959 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-a8660e92-7d3f-423f-b1f3-7f578986a5b3 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.129s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:35.598651 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Routers {'10aeccef-e762-4811-a411-a1c9817db442'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:24:35.599959 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:24:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['10aeccef-e762-4811-a411-a1c9817db442'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:35.645120 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:35.647832 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:35.668327 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.provisioning_blocks [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Transition to ACTIVE for port object 7dc69379-dd61-4add-8e0a-4ef00a211a0a will not be triggered until provisioned by entity DHCP. {{(pid=91643) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:24:35.700561 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:35.702573 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-8071551', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-8053391'] for router, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:35.711133 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-32c593ea-bfd1-444e-8c09-34ea7e26c59a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:35.759744 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 4f7409b5-c762-4162-ae2d-d2066230c4f4 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:35.760267 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:35.783762 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Network 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:24:35.786655 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:24:35.789125 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:24:35.829144 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Pushing event updated for resources: {'Port': ['ID=7dc69379-dd61-4add-8e0a-4ef00a211a0a,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:35.836618 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.125s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:35.886793 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-8003132'] for router_interface, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:36.079622 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] Set router 10aeccef-e762-4811-a411-a1c9817db442 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:24:36.174538 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Router 4969b258-f606-4def-b666-09764c475749 is scheduled to L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 with binding_index 1 {{(pid=91644) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 20 04:24:36.186829 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-63ec7f60-f881-4f36-91ef-ecb83ceb4f44 None None] Provisioning for port 7dc69379-dd61-4add-8e0a-4ef00a211a0a completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:24:36.196196 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-63ec7f60-f881-4f36-91ef-ecb83ceb4f44 None None] Provisioning complete for port 7dc69379-dd61-4add-8e0a-4ef00a211a0a triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:36.196580 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-63ec7f60-f881-4f36-91ef-ecb83ceb4f44 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:36.286206 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:36.336312 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-63ec7f60-f881-4f36-91ef-ecb83ceb4f44 None None] Port 7dc69379-dd61-4add-8e0a-4ef00a211a0a cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:24:36.348851 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-744c9302-f99e-4cdd-8a54-45a144279288 None None] Network 14c537db-01aa-4360-a348-47f94c75e03b requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:24:36.389101 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:24:36.427341 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Notify callbacks [] for router, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:36.429182 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] 10.222.0.39 "POST /v2.0/routers HTTP/1.1" status: 201 len: 922 time: 3.5374160 May 20 04:24:36.431986 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 37028) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:36.437330 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b279dab4-9c76-4ee7-9220-d7d8e4e154f0 req-7175ff85-8897-4638-b64b-10fd10251e9a tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:36.444608 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b279dab4-9c76-4ee7-9220-d7d8e4e154f0 req-7175ff85-8897-4638-b64b-10fd10251e9a tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 944 time: 0.3998132 May 20 04:24:36.486065 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Sync routers for ids ['4969b258-f606-4def-b666-09764c475749'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:24:36.511476 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33'}], 'enable_snat': True}, 'gw_port_id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:24:32Z', 'revision_number': 3, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port': {'id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:64:1f', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_gateway', 'standard_attr_id': 295, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:31Z', 'updated_at': '2021-05-20T04:24:35Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:36.516208 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3821ae85-bc4f-4200-977e-5781f4501509 None None] The sync data for ids ['10aeccef-e762-4811-a411-a1c9817db442'] in dv-cinder-782405-9-iscsi is: [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33'}], 'enable_snat': True}, 'gw_port_id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:24:32Z', 'revision_number': 3, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port': {'id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:64:1f', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_gateway', 'standard_attr_id': 295, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:31Z', 'updated_at': '2021-05-20T04:24:35Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}] {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:24:36.541116 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Routers {'4969b258-f606-4def-b666-09764c475749'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:24:36.541682 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:24:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['4969b258-f606-4def-b666-09764c475749'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:36.555099 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b279dab4-9c76-4ee7-9220-d7d8e4e154f0 req-39f15fce-bf99-4aab-b711-d5dcd02ab611 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1096163 May 20 04:24:36.594131 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-8d7c0303-a603-4ea9-a079-08b39409da86 None None] Provisioning for port cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:24:36.623498 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-8d7c0303-a603-4ea9-a079-08b39409da86 None None] Provisioning complete for port cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:36.623874 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-8d7c0303-a603-4ea9-a079-08b39409da86 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:36.682559 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3dab980d-d49d-4884-ab2d-abdacd724cb7 req-ac1b732f-19b8-4f66-b68a-3075dca9296f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:36.684235 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3dab980d-d49d-4884-ab2d-abdacd724cb7 req-ac1b732f-19b8-4f66-b68a-3075dca9296f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.2161579 May 20 04:24:36.736199 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Notify callbacks [] for port, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:36.748957 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:36.778253 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:36.847006 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3dab980d-d49d-4884-ab2d-abdacd724cb7 req-b158e66f-3225-435d-93b4-96d5e7dbbd95 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1462753 May 20 04:24:36.961271 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Set router 4969b258-f606-4def-b666-09764c475749 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:24:37.004798 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-7924897', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-8003163'] for router_interface, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:37.017381 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Pushing event updated for resources: {'Port': ['ID=7dc69379-dd61-4add-8e0a-4ef00a211a0a,revision_number=2']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:37.040045 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.262s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:37.179620 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Network 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:24:37.182433 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:24:37.215877 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-8d7c0303-a603-4ea9-a079-08b39409da86 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:37.335294 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device c3a0d63d-eb15-4b80-a8a4-15678c28b534 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:24:37.423642 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:37.433562 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-8d7c0303-a603-4ea9-a079-08b39409da86 None None] For port cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:24:37.453025 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-8d7c0303-a603-4ea9-a079-08b39409da86 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:37.460215 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:37.546399 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Router 10aeccef-e762-4811-a411-a1c9817db442 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91643) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:24:37.622998 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8d7c0303-a603-4ea9-a079-08b39409da86 None None] Pushing event updated for resources: {'Port': ['ID=cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:37.638035 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-8d7c0303-a603-4ea9-a079-08b39409da86 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.178s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:37.645146 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-78918fde-6db0-4a87-b8c5-12b82c06d2da None None] Provisioning complete for port 7dc69379-dd61-4add-8e0a-4ef00a211a0a triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:37.645854 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-78918fde-6db0-4a87-b8c5-12b82c06d2da None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:37.691650 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Allocated IP 172.20.0.1 (14c537db-01aa-4360-a348-47f94c75e03b/cdea475c-2929-4ae4-9271-87cc2fb9a378/8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:24:37.699403 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port c3a0d63d-eb15-4b80-a8a4-15678c28b534 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:24:37.720448 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port c3a0d63d-eb15-4b80-a8a4-15678c28b534 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:37.720973 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:37.763838 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:34Z', 'revision_number': 3, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:33Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:37.774976 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-78918fde-6db0-4a87-b8c5-12b82c06d2da None None] Port 7dc69379-dd61-4add-8e0a-4ef00a211a0a cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:24:37.782902 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Checking router: 4969b258-f606-4def-b666-09764c475749 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:24:37.783455 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Updating router 4969b258-f606-4def-b666-09764c475749 port 8f9c794c-3d7c-41d8-8927-b7ccdfe087ae binding host dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 20 04:24:37.846310 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-78918fde-6db0-4a87-b8c5-12b82c06d2da None None] Provisioning complete for port cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:37.846536 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:24:37.847005 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-78918fde-6db0-4a87-b8c5-12b82c06d2da None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:37.861698 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] 10.222.0.39 "PUT /v2.0/routers/10aeccef-e762-4811-a411-a1c9817db442/add_router_interface HTTP/1.1" status: 200 len: 503 time: 4.0061133 May 20 04:24:37.908422 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:37.916439 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-70ddae3f-960a-4517-ae1b-e0d9e97e3e70 req-434c1a2f-8ce1-4354-9338-4be4000f6923 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:37.927067 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-70ddae3f-960a-4517-ae1b-e0d9e97e3e70 req-434c1a2f-8ce1-4354-9338-4be4000f6923 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 944 time: 0.1565194 May 20 04:24:37.960824 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:37.988116 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Transition to ACTIVE for port object 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 will not be triggered until provisioned by entity DHCP. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:24:38.013506 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-70ddae3f-960a-4517-ae1b-e0d9e97e3e70 req-850e47d9-f615-456c-a426-c09b605fe6a5 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0850029 May 20 04:24:38.047268 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.055819 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:38.125699 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Network 14c537db-01aa-4360-a348-47f94c75e03b is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:24:38.129339 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:24:38.133075 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:24:38.169210 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ce5c1e7e-3fc6-403b-b0b2-7ff16ac73b1f req-3eed4f64-ef07-4306-9da8-51d42495d78f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:38.170941 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ce5c1e7e-3fc6-403b-b0b2-7ff16ac73b1f req-3eed4f64-ef07-4306-9da8-51d42495d78f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1160836 May 20 04:24:38.184701 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Pushing event updated for resources: {'Port': ['ID=8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:38.188647 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.197923 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.142s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:38.238003 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-8003132'] for router_interface, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.240884 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port c3a0d63d-eb15-4b80-a8a4-15678c28b534, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=c3a0d63d-eb15-4b80-a8a4-15678c28b534,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:24:38.246642 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ce5c1e7e-3fc6-403b-b0b2-7ff16ac73b1f req-a0da450a-4d55-4a94-ad0a-5fc3ce5a450e tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0689619 May 20 04:24:38.349847 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.364514 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:38.447383 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=c3a0d63d-eb15-4b80-a8a4-15678c28b534,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:38.447993 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.450455 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:38.526179 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b6bee055-19d1-4e07-bd15-16290eac96d9 up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:24:38.577180 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 37050) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:38.582880 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-3377ee61-7669-4183-bff8-18f4c142cb1e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Request body: {'security_group': {'name': 'tempest-securitygroup--1828126511', 'description': 'tempest-description--679793144'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:38.588598 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-3377ee61-7669-4183-bff8-18f4c142cb1e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:38.592747 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-3377ee61-7669-4183-bff8-18f4c142cb1e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Usage tracker for resource:security_group and tenant:198bc8ab083f4a729f32b8ef27a6ee77 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:38.595626 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-3377ee61-7669-4183-bff8-18f4c142cb1e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:38.601858 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-3377ee61-7669-4183-bff8-18f4c142cb1e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:38.612577 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3377ee61-7669-4183-bff8-18f4c142cb1e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.633122 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-3377ee61-7669-4183-bff8-18f4c142cb1e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:38.642721 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-3377ee61-7669-4183-bff8-18f4c142cb1e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Usage tracker for resource:security_group_rule and tenant:198bc8ab083f4a729f32b8ef27a6ee77 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:38.646861 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-3377ee61-7669-4183-bff8-18f4c142cb1e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:38.652538 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-d4b8b195-e81f-40c6-9731-a778f0d382df None None] Provisioning for port 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:24:38.653078 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-3377ee61-7669-4183-bff8-18f4c142cb1e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:38.656524 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Notify callbacks [] for port, precommit_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.669984 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-d4b8b195-e81f-40c6-9731-a778f0d382df None None] Provisioning complete for port 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:38.670359 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-d4b8b195-e81f-40c6-9731-a778f0d382df None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.681731 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Sync routers for ids ['10aeccef-e762-4811-a411-a1c9817db442'] in dv-cinder-782405-9-iscsi {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:24:38.700446 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Routers {'10aeccef-e762-4811-a411-a1c9817db442'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:24:38.701372 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:24:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['10aeccef-e762-4811-a411-a1c9817db442'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:38.701510 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Transition to ACTIVE for port object 8f9c794c-3d7c-41d8-8927-b7ccdfe087ae will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:24:38.731067 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.739252 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:38.743427 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.749395 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.756331 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3377ee61-7669-4183-bff8-18f4c142cb1e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.762201 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3377ee61-7669-4183-bff8-18f4c142cb1e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.766381 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-8cbff7f5-95bc-4c6d-9de1-ee6f26366cf7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:38.782809 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:38.799115 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-d4b8b195-e81f-40c6-9731-a778f0d382df None None] Port 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:24:38.805149 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port b6bee055-19d1-4e07-bd15-16290eac96d9 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:24:38.815886 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3377ee61-7669-4183-bff8-18f4c142cb1e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Pushing event updated for resources: {'SecurityGroup': ['ID=a510b283-ab41-46f8-afd9-15c93cb76abc,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:38.818859 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-3377ee61-7669-4183-bff8-18f4c142cb1e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:38.820569 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3377ee61-7669-4183-bff8-18f4c142cb1e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.821988 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-3377ee61-7669-4183-bff8-18f4c142cb1e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1779 time: 0.2437801 May 20 04:24:38.825148 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 37054) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:38.830482 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-9794c65b-e1a5-4b13-a066-c7e72f39e06a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Request body: {'security_group_rule': {'security_group_id': 'a510b283-ab41-46f8-afd9-15c93cb76abc', 'protocol': 'tcp', 'ethertype': 'IPv4', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:38.831353 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port b6bee055-19d1-4e07-bd15-16290eac96d9 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:38.831743 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.837781 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-9794c65b-e1a5-4b13-a066-c7e72f39e06a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:38.841310 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-9794c65b-e1a5-4b13-a066-c7e72f39e06a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Usage tracker for resource:security_group_rule and tenant:198bc8ab083f4a729f32b8ef27a6ee77 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:38.844721 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-9794c65b-e1a5-4b13-a066-c7e72f39e06a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:38.850979 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-9794c65b-e1a5-4b13-a066-c7e72f39e06a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:38.870301 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9794c65b-e1a5-4b13-a066-c7e72f39e06a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.890254 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Attempting to bind port 8f9c794c-3d7c-41d8-8927-b7ccdfe087ae on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:24:38.890561 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Attempting to bind port 8f9c794c-3d7c-41d8-8927-b7ccdfe087ae by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:24:38.890936 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Attempting to bind port 8f9c794c-3d7c-41d8-8927-b7ccdfe087ae on network 712ce8c4-8feb-4719-bf13-e06992125cd9 {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:24:38.902000 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-7924897', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-8003163'] for router_interface, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.905771 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 24, 15), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:24:38.906148 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Checking segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:24:38.906562 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.907569 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Bound using segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:24:38.907569 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Bound port: 8f9c794c-3d7c-41d8-8927-b7ccdfe087ae, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:24:38.907967 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.909420 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Pushing event updated for resources: {'Port': ['ID=8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:38.914293 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.132s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:38.936315 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-08b47d33-b480-4f94-917f-e18e021c1929 None None] Pushing event updated for resources: {'Port': ['ID=8f9c794c-3d7c-41d8-8927-b7ccdfe087ae,revision_number=2']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:38.947888 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9794c65b-e1a5-4b13-a066-c7e72f39e06a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.953811 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9794c65b-e1a5-4b13-a066-c7e72f39e06a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:38.957283 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-08b47d33-b480-4f94-917f-e18e021c1929 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.218s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:38.959468 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:38.977096 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Set router 10aeccef-e762-4811-a411-a1c9817db442 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:24:39.000955 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9794c65b-e1a5-4b13-a066-c7e72f39e06a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=20477e4c-1780-44d0-ac9e-bc3e144299e4,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:39.008490 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9794c65b-e1a5-4b13-a066-c7e72f39e06a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:39.010758 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9794c65b-e1a5-4b13-a066-c7e72f39e06a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:39.015267 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-9794c65b-e1a5-4b13-a066-c7e72f39e06a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.1893010 May 20 04:24:39.016206 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 37058) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:39.025473 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-b07a5335-eff0-4f6d-a7e2-918d5e7ad195 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Request body: {'security_group_rule': {'security_group_id': 'a510b283-ab41-46f8-afd9-15c93cb76abc', 'protocol': 'icmp', 'ethertype': 'IPv4', 'direction': 'ingress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:39.032529 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-b07a5335-eff0-4f6d-a7e2-918d5e7ad195 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:39.038339 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-b07a5335-eff0-4f6d-a7e2-918d5e7ad195 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Usage tracker for resource:security_group_rule and tenant:198bc8ab083f4a729f32b8ef27a6ee77 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:39.041612 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-b07a5335-eff0-4f6d-a7e2-918d5e7ad195 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:39.049448 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-b07a5335-eff0-4f6d-a7e2-918d5e7ad195 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:39.050790 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] For port 8f9c794c-3d7c-41d8-8927-b7ccdfe087ae, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:24:39.071604 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] For port 8f9c794c-3d7c-41d8-8927-b7ccdfe087ae, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=8f9c794c-3d7c-41d8-8927-b7ccdfe087ae,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:24:39.072025 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Network 14c537db-01aa-4360-a348-47f94c75e03b is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:24:39.073627 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:24:39.088488 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:39.093050 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b07a5335-eff0-4f6d-a7e2-918d5e7ad195 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:39.101875 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:39.109641 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:39.242321 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'ad654613-8b28-495c-9d73-8e92df6e2464', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b6bee055-19d1-4e07-bd15-16290eac96d9'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:24:39.250364 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Routers {'4969b258-f606-4def-b666-09764c475749'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:24:39.250999 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:24:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['4969b258-f606-4def-b666-09764c475749'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:39.336552 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4d7b933a-d66c-41c2-8719-c648474164a1 req-2ef1a0a5-b228-4b83-a9f5-5fc19cd92b08 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:39.344863 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4d7b933a-d66c-41c2-8719-c648474164a1 req-2ef1a0a5-b228-4b83-a9f5-5fc19cd92b08 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 944 time: 0.1714258 May 20 04:24:39.348456 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:24:39.359425 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Router 4969b258-f606-4def-b666-09764c475749 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91644) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:24:39.377895 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b07a5335-eff0-4f6d-a7e2-918d5e7ad195 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:39.411182 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b6bee055-19d1-4e07-bd15-16290eac96d9, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b6bee055-19d1-4e07-bd15-16290eac96d9,segment=NetworkSegment(6fe99325-a72c-48bf-abc7-9bbcc4b753ff),segment_id=6fe99325-a72c-48bf-abc7-9bbcc4b753ff)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:24:39.416838 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4d7b933a-d66c-41c2-8719-c648474164a1 req-e55cb50f-28b3-4dd3-9e15-fb840757be28 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0678673 May 20 04:24:39.419199 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b07a5335-eff0-4f6d-a7e2-918d5e7ad195 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:39.431661 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:39.442076 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-08b47d33-b480-4f94-917f-e18e021c1929 None None] Pushing event updated for resources: {'Port': ['ID=8f9c794c-3d7c-41d8-8927-b7ccdfe087ae,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:39.444845 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:39.457460 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-08b47d33-b480-4f94-917f-e18e021c1929 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.348s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:39.458023 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-08b47d33-b480-4f94-917f-e18e021c1929 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:39.476882 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b07a5335-eff0-4f6d-a7e2-918d5e7ad195 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=79b6e7ea-c5fa-458b-9584-4c39ebfa2404,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:39.479383 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-b07a5335-eff0-4f6d-a7e2-918d5e7ad195 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:39.491106 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33'}], 'enable_snat': True}, 'gw_port_id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:24:36Z', 'revision_number': 4, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port': {'id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:64:1f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_gateway', 'standard_attr_id': 295, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:31Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '7dc69379-dd61-4add-8e0a-4ef00a211a0a', 'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'mac_address': 'fa:16:3e:a4:49:d4', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_interface', 'standard_attr_id': 307, 'fixed_ips': [{'subnet_id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:35Z', 'updated_at': '2021-05-20T04:24:36Z', 'revision_number': 2, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'subnets': [{'id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:39.493734 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b07a5335-eff0-4f6d-a7e2-918d5e7ad195 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:39.497197 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Checking router: 10aeccef-e762-4811-a411-a1c9817db442 for host: dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:24:39.497762 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Updating router 10aeccef-e762-4811-a411-a1c9817db442 port 7dc69379-dd61-4add-8e0a-4ef00a211a0a binding host dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 20 04:24:39.498480 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-b07a5335-eff0-4f6d-a7e2-918d5e7ad195 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.4802694 May 20 04:24:39.499374 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 37068) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:39.515641 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Request body: {'floatingip': {'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:39.521765 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:39.524261 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'ad654613-8b28-495c-9d73-8e92df6e2464', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b6bee055-19d1-4e07-bd15-16290eac96d9', 'code': 200} May 20 04:24:39.530106 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Usage tracker for resource:floatingip and tenant:198bc8ab083f4a729f32b8ef27a6ee77 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:39.538950 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:39.548338 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:39.565542 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for floatingip, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:39.569281 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:39.621937 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:24:39.640577 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] 10.222.0.39 "PUT /v2.0/routers/4969b258-f606-4def-b666-09764c475749/add_router_interface HTTP/1.1" status: 200 len: 503 time: 3.2071941 May 20 04:24:39.709230 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-3efe4275-01ee-464e-91bf-794938389356 None None] Provisioning complete for port 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:39.709807 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-3efe4275-01ee-464e-91bf-794938389356 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:39.769970 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:39.770348 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:39.790177 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a1d220ef-dc5d-4ff4-a0cb-48407c75654f req-43ddc898-7050-4603-b7bc-95fa1656958a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:39.791680 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a1d220ef-dc5d-4ff4-a0cb-48407c75654f req-43ddc898-7050-4603-b7bc-95fa1656958a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1288402 May 20 04:24:39.829625 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] Set router 4969b258-f606-4def-b666-09764c475749 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:24:39.856607 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a1d220ef-dc5d-4ff4-a0cb-48407c75654f req-756413c0-b0af-4e61-81b3-2aeb6671440d tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0585608 May 20 04:24:39.875312 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:39.913810 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=b6bee055-19d1-4e07-bd15-16290eac96d9,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:39.936658 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.478s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:39.941184 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-3efe4275-01ee-464e-91bf-794938389356 None None] Port 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:24:40.037860 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Notify callbacks [] for port, precommit_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:40.041116 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Allocated IP 172.27.0.159 (712ce8c4-8feb-4719-bf13-e06992125cd9/e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5/4af94d38-0efa-4427-9792-914bf823abf2) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:24:40.054131 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.provisioning_blocks [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Transition to ACTIVE for port object 7dc69379-dd61-4add-8e0a-4ef00a211a0a will not be triggered until provisioned by entity L2. {{(pid=91646) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:24:40.062584 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:40.067832 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-a8660e92-7d3f-423f-b1f3-7f578986a5b3 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:40.168660 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-148bf171-b3de-45b9-9dac-599a88dbdb32 None None] Pushing event updated for resources: {'Port': ['ID=7dc69379-dd61-4add-8e0a-4ef00a211a0a,revision_number=3']} {{(pid=91646) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:40.172767 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-148bf171-b3de-45b9-9dac-599a88dbdb32 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.105s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:40.316701 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device c3a0d63d-eb15-4b80-a8a4-15678c28b534 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:24:40.366556 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Attempting to bind port 7dc69379-dd61-4add-8e0a-4ef00a211a0a on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:24:40.366918 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Attempting to bind port 7dc69379-dd61-4add-8e0a-4ef00a211a0a by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': 'b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 808, 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d'}] {{(pid=91646) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:24:40.367459 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Attempting to bind port 7dc69379-dd61-4add-8e0a-4ef00a211a0a on network 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:24:40.373528 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 24, 15), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:24:40.373903 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Checking segment: {'id': 'b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 808, 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91646) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:24:40.374301 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:40.374826 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Bound using segment: {'id': 'b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 808, 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d'} {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:24:40.375334 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Bound port: 7dc69379-dd61-4add-8e0a-4ef00a211a0a, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 808, 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d'}}] {{(pid=91646) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:24:40.375745 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:40.383356 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 37088) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:40.391037 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Request body: {'network': {'name': 'tempest-TestVolumeBootPattern-1346048449-network', 'tenant_id': '207fc367830344daad06243fb26256f7'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:40.397799 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:40.408068 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Usage tracker for resource:network and tenant:207fc367830344daad06243fb26256f7 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:40.411771 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:39Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 2, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:40.414636 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:40.418433 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-89c78b17-ddd5-4a34-9841-c507c3bcafaf None None] The sync data for ids ['4969b258-f606-4def-b666-09764c475749'] in dv-cinder-782405-9-iscsi is: [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:39Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 2, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:24:40.424717 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:40.424836 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port c3a0d63d-eb15-4b80-a8a4-15678c28b534 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:40.425106 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:40.431369 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:40.445071 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Made reservation on behalf of 207fc367830344daad06243fb26256f7 for: {'network': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:24:40.447550 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-585852'] for network, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:40.471806 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:40.481913 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.db [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] For port 7dc69379-dd61-4add-8e0a-4ef00a211a0a, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91646) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:24:40.485874 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.db [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] For port 7dc69379-dd61-4add-8e0a-4ef00a211a0a, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=7dc69379-dd61-4add-8e0a-4ef00a211a0a,segment=NetworkSegment(b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7),segment_id=b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7)] {{(pid=91646) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:24:40.491089 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:40.493266 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:40.499414 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Usage tracker for resource:security_group_rule and tenant:207fc367830344daad06243fb26256f7 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:40.502410 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-a8660e92-7d3f-423f-b1f3-7f578986a5b3 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:40.503441 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:40.510890 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:40.538945 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:40.555371 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-04de94b9-aee2-4cc5-a626-a6072e5e5018 tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:40.593274 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-148bf171-b3de-45b9-9dac-599a88dbdb32 None None] Pushing event updated for resources: {'Port': ['ID=7dc69379-dd61-4add-8e0a-4ef00a211a0a,revision_number=4']} {{(pid=91646) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:40.598586 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-148bf171-b3de-45b9-9dac-599a88dbdb32 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:40.604813 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Routers {'10aeccef-e762-4811-a411-a1c9817db442'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:24:40.605127 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:24:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['10aeccef-e762-4811-a411-a1c9817db442'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:40.670369 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:24:40.682220 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:24:40.701829 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:24:40.789154 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Pushing event updated for resources: {'Port': ['ID=4af94d38-0efa-4427-9792-914bf823abf2,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:40.798640 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.243s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:40.872368 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] Set router 10aeccef-e762-4811-a411-a1c9817db442 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:24:40.889837 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:40.900654 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ef535eb4-73b5-4f37-9259-23cf87245fb6 req-1e144deb-d8f9-4cfe-a2e2-0aa93449e4c5 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:40.902199 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ef535eb4-73b5-4f37-9259-23cf87245fb6 req-1e144deb-d8f9-4cfe-a2e2-0aa93449e4c5 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 945 time: 0.3084090 May 20 04:24:40.905203 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:40.909716 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-798267d7-8751-47c3-bd5e-7a231208c725 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:40.924420 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for floatingip, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:40.988755 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 810} {{(pid=91643) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 20 04:24:41.023774 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=66fa43f5-dfa4-437d-b177-e6992999a970,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:41.038924 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:41.039285 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.segments_db [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Added segment 3bd4c975-0dc6-43e6-b17a-8e2fa22a9507 of type vlan for network b5c87200-df2c-432b-8fc4-24090c6d30f3 May 20 04:24:41.040579 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.131s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:41.049667 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ef535eb4-73b5-4f37-9259-23cf87245fb6 req-b58c457a-8d22-4b32-bc39-407c642fed3f tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1327372 May 20 04:24:41.077689 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:41.130591 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8760688516720'] for network, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:41.322014 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33'}], 'enable_snat': True}, 'gw_port_id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:24:36Z', 'revision_number': 4, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port': {'id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:64:1f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_gateway', 'standard_attr_id': 295, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:31Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '7dc69379-dd61-4add-8e0a-4ef00a211a0a', 'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'mac_address': 'fa:16:3e:a4:49:d4', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_interface', 'standard_attr_id': 307, 'fixed_ips': [{'subnet_id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:35Z', 'updated_at': '2021-05-20T04:24:40Z', 'revision_number': 4, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'subnets': [{'id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:41.330866 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-80ce6433-104e-4187-ae97-27c53f8de228 None None] The sync data for ids ['10aeccef-e762-4811-a411-a1c9817db442'] in dv-cinder-782405-9-iscsi is: [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33'}], 'enable_snat': True}, 'gw_port_id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:24:36Z', 'revision_number': 4, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port': {'id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:64:1f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_gateway', 'standard_attr_id': 295, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:31Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '7dc69379-dd61-4add-8e0a-4ef00a211a0a', 'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'mac_address': 'fa:16:3e:a4:49:d4', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_interface', 'standard_attr_id': 307, 'fixed_ips': [{'subnet_id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:35Z', 'updated_at': '2021-05-20T04:24:40Z', 'revision_number': 4, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'subnets': [{'id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:24:41.340063 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915'] for network, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:41.356356 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:41.356954 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:41.364598 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-5c761957-b070-48e2-a819-b03a6397528e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:41.515392 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Pushing event updated for resources: {'Network': ['ID=b5c87200-df2c-432b-8fc4-24090c6d30f3,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:41.518137 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.154s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:41.525543 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Network b5c87200-df2c-432b-8fc4-24090c6d30f3 details requested by agent ovs-agent-dv-cinder-782405-9-iscsi with host dv-cinder-782405-9-iscsi {{(pid=91645) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 20 04:24:41.542572 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] 10.222.0.39 "POST /v2.0/networks HTTP/1.1" status: 201 len: 885 time: 1.1578689 May 20 04:24:41.545686 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 37098) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:41.553565 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-c1cee381-6947-40f4-aed7-216aa79f9601 admin admin] Request body: {'subnet': {'network_id': 'b5c87200-df2c-432b-8fc4-24090c6d30f3', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestVolumeBootPattern-1346048449-subnet', 'tenant_id': '207fc367830344daad06243fb26256f7', 'ip_version': 4}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:41.560271 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-c1cee381-6947-40f4-aed7-216aa79f9601 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:41.562789 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for port, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:41.565112 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-c1cee381-6947-40f4-aed7-216aa79f9601 admin admin] Usage tracker for resource:subnet and tenant:207fc367830344daad06243fb26256f7 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:41.573712 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-c1cee381-6947-40f4-aed7-216aa79f9601 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:41.579144 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-17a57290-1c55-4a58-9316-a627b239de93 req-fec4d5cb-527f-4034-bbd3-fea4f3812023 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:41.585679 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-c1cee381-6947-40f4-aed7-216aa79f9601 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:41.588592 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-17a57290-1c55-4a58-9316-a627b239de93 req-fec4d5cb-527f-4034-bbd3-fea4f3812023 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.4035130 May 20 04:24:41.604252 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:41.609939 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:41.665205 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-c1cee381-6947-40f4-aed7-216aa79f9601 admin admin] Made reservation on behalf of 207fc367830344daad06243fb26256f7 for: {'subnet': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:24:41.667622 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c1cee381-6947-40f4-aed7-216aa79f9601 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-1810190'] for subnet, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:41.771852 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-17a57290-1c55-4a58-9316-a627b239de93 req-f0369fc2-651f-4586-aecb-5007fa2d79fd tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1822686 May 20 04:24:41.781165 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Pushing event updated for resources: {'Port': ['ID=4af94d38-0efa-4427-9792-914bf823abf2,revision_number=2']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:41.784447 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.175s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:41.798476 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-8003251'] for floatingip, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:41.819825 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:41.821178 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 693 time: 2.3208785 May 20 04:24:41.827536 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-23a38370-5bc1-4266-9593-5511b4ec2c99 req-18fb397a-e125-4a6a-b65c-5993bd2a9ffd service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:41.829011 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-23a38370-5bc1-4266-9593-5511b4ec2c99 req-18fb397a-e125-4a6a-b65c-5993bd2a9ffd service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 1098 time: 0.2316587 May 20 04:24:41.927406 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-23a38370-5bc1-4266-9593-5511b4ec2c99 req-fcfd5ccd-c406-4c38-8e3e-04b1872a469d service neutron] Synchronizing usage tracker for tenant:11f8ac5c97ce4c03adee5db9032a8fb4 on resource:security_group {{(pid=91644) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:24:42.064681 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-23a38370-5bc1-4266-9593-5511b4ec2c99 req-fcfd5ccd-c406-4c38-8e3e-04b1872a469d service neutron] Unset dirty status for tenant:11f8ac5c97ce4c03adee5db9032a8fb4 on resource:security_group {{(pid=91644) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:24:42.065785 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-23a38370-5bc1-4266-9593-5511b4ec2c99 req-fcfd5ccd-c406-4c38-8e3e-04b1872a469d service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2292104 May 20 04:24:42.089349 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-bfcb57fa-7d58-40de-8233-7634327cde24 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Synchronizing usage tracker for tenant:198bc8ab083f4a729f32b8ef27a6ee77 on resource:security_group {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:24:42.113793 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-bfcb57fa-7d58-40de-8233-7634327cde24 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Unset dirty status for tenant:198bc8ab083f4a729f32b8ef27a6ee77 on resource:security_group {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:24:42.115464 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-bfcb57fa-7d58-40de-8233-7634327cde24 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?name=tempest-securitygroup--1828126511&fields=id&tenant_id=198bc8ab083f4a729f32b8ef27a6ee77 HTTP/1.1" status: 200 len: 267 time: 0.0803564 May 20 04:24:42.277870 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c1cee381-6947-40f4-aed7-216aa79f9601 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-1988978', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-1810236'] for subnet, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:42.281968 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-31ca805b-6bf6-4f9c-b1a9-171a1016b2a8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:42.341567 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-4ed541cc-9558-426c-a059-1d954759039e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:42.343365 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-4ed541cc-9558-426c-a059-1d954759039e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/networks?id=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d HTTP/1.1" status: 200 len: 809 time: 0.2110908 May 20 04:24:42.355373 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b08da7e5-5994-4ffa-8345-ee2857f3d9e2 req-72da6440-111f-49ef-a268-62b1cda038fa tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:42.356799 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b08da7e5-5994-4ffa-8345-ee2857f3d9e2 req-72da6440-111f-49ef-a268-62b1cda038fa tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 945 time: 0.1780548 May 20 04:24:42.358916 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c1cee381-6947-40f4-aed7-216aa79f9601 admin admin] Pushing event updated for resources: {'Subnet': ['ID=2927e09a-9033-4dd7-a313-77a3b77cd585,revision_number=0']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:42.361953 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-c1cee381-6947-40f4-aed7-216aa79f9601 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.080s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:42.374568 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c1cee381-6947-40f4-aed7-216aa79f9601 admin admin] Checking segment: {'id': '3bd4c975-0dc6-43e6-b17a-8e2fa22a9507', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 810, 'network_id': 'b5c87200-df2c-432b-8fc4-24090c6d30f3'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:24:42.374910 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c1cee381-6947-40f4-aed7-216aa79f9601 admin admin] Network hostable DHCP agents. Network: b5c87200-df2c-432b-8fc4-24090c6d30f3, hostable agents: ['fffae25a-f450-4182-8f9e-ac29d143f6c9'], hosted agents: [] {{(pid=91644) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 20 04:24:42.383979 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-f327bb91-9042-40e5-a23a-c436263f453c tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/quotas/198bc8ab083f4a729f32b8ef27a6ee77 HTTP/1.1" status: 200 len: 375 time: 0.0261128 May 20 04:24:42.393119 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c1cee381-6947-40f4-aed7-216aa79f9601 admin admin] Network b5c87200-df2c-432b-8fc4-24090c6d30f3 is scheduled to be hosted by DHCP agent fffae25a-f450-4182-8f9e-ac29d143f6c9 with binding_index 1 {{(pid=91644) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 20 04:24:42.403258 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-c1cee381-6947-40f4-aed7-216aa79f9601 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-782405-9-iscsi, b5c87200-df2c-432b-8fc4-24090c6d30f3) {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:24:42.411739 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-c1cee381-6947-40f4-aed7-216aa79f9601 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:42.412066 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-8fa6bc03-984d-40bb-9a56-cfcf0d4d0d35 None None] Network b5c87200-df2c-432b-8fc4-24090c6d30f3 requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:24:42.412333 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c1cee381-6947-40f4-aed7-216aa79f9601 admin admin] Notify callbacks [] for subnet, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:42.433969 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-c1cee381-6947-40f4-aed7-216aa79f9601 admin admin] 10.222.0.39 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 848 time: 0.8864655 May 20 04:24:42.436511 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 37120) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:42.442248 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Request body: {'router': {'name': 'tempest-TestVolumeBootPattern-1346048449-router', 'tenant_id': '207fc367830344daad06243fb26256f7', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:42.446493 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:42.450789 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Usage tracker for resource:router and tenant:207fc367830344daad06243fb26256f7 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:42.454442 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:42.460744 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:42.470023 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b08da7e5-5994-4ffa-8345-ee2857f3d9e2 req-5dcbc806-e95d-480d-aa4a-1d3144fcee36 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1011388 May 20 04:24:42.485311 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-8071540', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-260712'] for router, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:42.486433 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-8056436', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-8053301', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-7881139', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-260707'] for router, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:42.505233 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:42.543002 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-21b839ad-dbd8-4b3b-802b-333f38103d8b tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Synchronizing usage tracker for tenant:198bc8ab083f4a729f32b8ef27a6ee77 on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:24:42.566985 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-21b839ad-dbd8-4b3b-802b-333f38103d8b tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Unset dirty status for tenant:198bc8ab083f4a729f32b8ef27a6ee77 on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:24:42.568818 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-21b839ad-dbd8-4b3b-802b-333f38103d8b tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=198bc8ab083f4a729f32b8ef27a6ee77&fields=id HTTP/1.1" status: 200 len: 348 time: 0.1790919 May 20 04:24:42.670055 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-5b961053-015c-4cff-97e0-adf9f69ed9c4 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:42.689375 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcp85c0b5c8-c799-5455-967d-3e9c2c8b4839-b5c87200-df2c-432b-8fc4-24090c6d30f3', 'network_id': 'b5c87200-df2c-432b-8fc4-24090c6d30f3', 'tenant_id': '207fc367830344daad06243fb26256f7', 'fixed_ips': [{'subnet_id': '2927e09a-9033-4dd7-a313-77a3b77cd585'}]}} from dv-cinder-782405-9-iscsi. {{(pid=91645) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 20 04:24:42.690548 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:42.724739 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-fae2f055-66f7-4b40-a7f9-6f64e3e11c98 service neutron] 10.222.0.39 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1421273 May 20 04:24:42.819771 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-8053308', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-8071430', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-260685'] for router, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:42.871098 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.db_base_plugin_common [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Allocated IP 172.20.0.2 (b5c87200-df2c-432b-8fc4-24090c6d30f3/2927e09a-9033-4dd7-a313-77a3b77cd585/83f0181b-202d-4c5d-ab40-d746ebe28872) {{(pid=91645) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:24:43.026390 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-3a1683dc-5853-4ca0-880a-9bd9686fdb0e service neutron] 10.222.0.39 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2953622 May 20 04:24:43.057372 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a01328b9-4073-4082-956c-72fae04c357b req-cd0e3cee-dacb-44f7-b2ae-3d416ecd8870 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:43.059708 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a01328b9-4073-4082-956c-72fae04c357b req-cd0e3cee-dacb-44f7-b2ae-3d416ecd8870 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1370983 May 20 04:24:43.139333 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a01328b9-4073-4082-956c-72fae04c357b req-c40506f1-5102-42fb-96b2-f546df91c18a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0704505 May 20 04:24:43.160045 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Notify callbacks [] for port, precommit_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:43.172496 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Transition to ACTIVE for port object 83f0181b-202d-4c5d-ab40-d746ebe28872 will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:24:43.199589 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Transition to ACTIVE for port object 83f0181b-202d-4c5d-ab40-d746ebe28872 will not be triggered until provisioned by entity DHCP. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:24:43.207740 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:43.237047 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:43.239085 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:43.271959 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:43.285319 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-76dde3e1-208b-4782-9386-e83d2dcf1c6a req-edc105d9-3a67-4ed7-bdaf-2cd30e6a087e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 210 time: 0.0510206 May 20 04:24:43.295767 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Network b5c87200-df2c-432b-8fc4-24090c6d30f3 is already hosted by enough agents. {{(pid=91645) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:24:43.299377 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.base_scheduler [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Resources bound (agent type, host, resource id): {{(pid=91645) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:24:43.302156 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Received port after_create {{(pid=91645) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:24:43.418606 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 7dc69379-dd61-4add-8e0a-4ef00a211a0a up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:24:43.449091 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Pushing event updated for resources: {'Port': ['ID=83f0181b-202d-4c5d-ab40-d746ebe28872,revision_number=1']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:43.457848 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Attempting to bind port 83f0181b-202d-4c5d-ab40-d746ebe28872 on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:24:43.458227 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Attempting to bind port 83f0181b-202d-4c5d-ab40-d746ebe28872 by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '3bd4c975-0dc6-43e6-b17a-8e2fa22a9507', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 810, 'network_id': 'b5c87200-df2c-432b-8fc4-24090c6d30f3'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:24:43.458590 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Attempting to bind port 83f0181b-202d-4c5d-ab40-d746ebe28872 on network b5c87200-df2c-432b-8fc4-24090c6d30f3 {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:24:43.459382 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Allocated IP 172.27.0.182 (712ce8c4-8feb-4719-bf13-e06992125cd9/e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5/0e5ce97c-8370-428d-b569-67d6ffe03e3b) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:24:43.460900 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.222s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:43.476504 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 24, 15), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:24:43.476907 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Checking segment: {'id': '3bd4c975-0dc6-43e6-b17a-8e2fa22a9507', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 810, 'network_id': 'b5c87200-df2c-432b-8fc4-24090c6d30f3'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:24:43.477311 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:43.477699 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Bound using segment: {'id': '3bd4c975-0dc6-43e6-b17a-8e2fa22a9507', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 810, 'network_id': 'b5c87200-df2c-432b-8fc4-24090c6d30f3'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:24:43.478171 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Bound port: 83f0181b-202d-4c5d-ab40-d746ebe28872, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '3bd4c975-0dc6-43e6-b17a-8e2fa22a9507', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 810, 'network_id': 'b5c87200-df2c-432b-8fc4-24090c6d30f3'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:24:43.478544 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:43.577063 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port 7dc69379-dd61-4add-8e0a-4ef00a211a0a completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:24:43.590837 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] For port 83f0181b-202d-4c5d-ab40-d746ebe28872, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:24:43.593001 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 7dc69379-dd61-4add-8e0a-4ef00a211a0a triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:43.593407 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:43.639850 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:43.645230 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] For port 83f0181b-202d-4c5d-ab40-d746ebe28872, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=83f0181b-202d-4c5d-ab40-d746ebe28872,segment=NetworkSegment(3bd4c975-0dc6-43e6-b17a-8e2fa22a9507),segment_id=3bd4c975-0dc6-43e6-b17a-8e2fa22a9507)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:24:43.651834 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:43.658298 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:43.682413 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:43.694481 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-48e82479-f761-4613-ba12-f812c81e859c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:43.777765 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:24:43.783247 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:24:43.789041 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:24:43.811053 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-deac9984-06ce-47a7-aa42-abb21d2bcc64 req-b83976ad-02b0-4b15-91be-9903eb97cbb3 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:43.820863 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Pushing event updated for resources: {'Port': ['ID=0e5ce97c-8370-428d-b569-67d6ffe03e3b,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:43.824093 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.130s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:43.831448 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-deac9984-06ce-47a7-aa42-abb21d2bcc64 req-b83976ad-02b0-4b15-91be-9903eb97cbb3 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 945 time: 0.2077003 May 20 04:24:43.895613 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Pushing event updated for resources: {'Port': ['ID=83f0181b-202d-4c5d-ab40-d746ebe28872,revision_number=2']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:43.918380 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.260s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:43.936770 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-deac9984-06ce-47a7-aa42-abb21d2bcc64 req-233eb963-598b-4d3a-884a-ed0129bf06f5 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1030815 May 20 04:24:43.978596 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:44.026811 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port 7dc69379-dd61-4add-8e0a-4ef00a211a0a, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=7dc69379-dd61-4add-8e0a-4ef00a211a0a,segment=NetworkSegment(b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7),segment_id=b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:24:44.035422 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:44.036346 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 8f9c794c-3d7c-41d8-8927-b7ccdfe087ae up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:24:44.089333 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:44.316550 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=7dc69379-dd61-4add-8e0a-4ef00a211a0a,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:44.345645 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.256s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:44.353140 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port 8f9c794c-3d7c-41d8-8927-b7ccdfe087ae completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:24:44.366863 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 8f9c794c-3d7c-41d8-8927-b7ccdfe087ae triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:44.367257 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:44.516908 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:44.575816 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5146978b-551f-4a49-a9e3-d8bb3bd5d332 req-4c21eb97-494e-4c0e-8f7f-183392448197 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 210 time: 0.0830429 May 20 04:24:44.594510 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Sync routers for ids ['4969b258-f606-4def-b666-09764c475749'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:24:44.606828 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-62f30f99-40ce-4500-8b7e-cd80550dd1cb req-4f80e205-0694-4e87-a5a0-6e00ddcd7124 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:44.608304 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-62f30f99-40ce-4500-8b7e-cd80550dd1cb req-4f80e205-0694-4e87-a5a0-6e00ddcd7124 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1830790 May 20 04:24:44.623625 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Routers {'4969b258-f606-4def-b666-09764c475749'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:24:44.624254 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:24:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['4969b258-f606-4def-b666-09764c475749'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:44.653711 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-62f30f99-40ce-4500-8b7e-cd80550dd1cb req-48ecfea0-660f-4d21-b4b4-f417e5e18da7 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0393100 May 20 04:24:44.695528 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:44.728979 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port 8f9c794c-3d7c-41d8-8927-b7ccdfe087ae, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=8f9c794c-3d7c-41d8-8927-b7ccdfe087ae,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:24:44.777655 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:44.786343 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-1aeae814-b2cf-41fb-8dec-30c2d92c73b8 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:44.877025 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Set router 4969b258-f606-4def-b666-09764c475749 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:24:44.889175 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-8071551', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-8053391'] for router, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:44.930476 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=8f9c794c-3d7c-41d8-8927-b7ccdfe087ae,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:44.941981 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.156s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:45.088933 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:45.239098 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5ff3669a-fa66-44fc-98fc-337cfd6f1de6 req-86954deb-6e45-4742-9767-10c1b6cbda44 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:45.241166 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5ff3669a-fa66-44fc-98fc-337cfd6f1de6 req-86954deb-6e45-4742-9767-10c1b6cbda44 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 945 time: 0.1300287 May 20 04:24:45.244797 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-a0fd4d33-e7d8-4ad0-a62c-80ccdfa486b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:45.325871 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e5fc1706-c552-4513-b5f5-e1c9e64b53a3 None None] Network b5c87200-df2c-432b-8fc4-24090c6d30f3 requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:24:45.383824 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5ff3669a-fa66-44fc-98fc-337cfd6f1de6 req-1f92a263-7587-4cf4-ae77-7036364eb44f tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1365724 May 20 04:24:45.411550 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-72fd4d5c-9d5f-47e3-98fb-12b5abf6bf07 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:45.431509 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-2c513b17-1b1c-4680-a53d-a59bd4b0f6cf None None] Provisioning for port 83f0181b-202d-4c5d-ab40-d746ebe28872 completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:24:45.434373 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-72fd4d5c-9d5f-47e3-98fb-12b5abf6bf07 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Synchronizing usage tracker for tenant:198bc8ab083f4a729f32b8ef27a6ee77 on resource:network {{(pid=91644) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:24:45.463619 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-72fd4d5c-9d5f-47e3-98fb-12b5abf6bf07 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Unset dirty status for tenant:198bc8ab083f4a729f32b8ef27a6ee77 on resource:network {{(pid=91644) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:24:45.464915 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-72fd4d5c-9d5f-47e3-98fb-12b5abf6bf07 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.74 "GET /v2.0/networks?id=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d HTTP/1.1" status: 200 len: 809 time: 0.3336074 May 20 04:24:45.471597 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 83f0181b-202d-4c5d-ab40-d746ebe28872 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:24:45.545847 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Router 5faa3599-06bc-45d7-8b3f-e21d104b087a is scheduled to L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 with binding_index 1 {{(pid=91644) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 20 04:24:45.553542 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-840ffbac-1de3-4deb-9966-3d9c8c94b1b0 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Synchronizing usage tracker for tenant:198bc8ab083f4a729f32b8ef27a6ee77 on resource:security_group {{(pid=91644) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:24:45.569003 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-840ffbac-1de3-4deb-9966-3d9c8c94b1b0 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Unset dirty status for tenant:198bc8ab083f4a729f32b8ef27a6ee77 on resource:security_group {{(pid=91644) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:24:45.570727 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-840ffbac-1de3-4deb-9966-3d9c8c94b1b0 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.74 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=198bc8ab083f4a729f32b8ef27a6ee77 HTTP/1.1" status: 200 len: 380 time: 0.0933774 May 20 04:24:45.624020 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:39Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 2, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:45.640580 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-65344260-99ea-419c-9f68-fed55e74a4c4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Request body: {'port': {'device_id': '9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'admin_state_up': True, 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'security_groups': ['a510b283-ab41-46f8-afd9-15c93cb76abc']}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:45.660248 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port 83f0181b-202d-4c5d-ab40-d746ebe28872 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:24:45.663154 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Checking router: 4969b258-f606-4def-b666-09764c475749 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:24:45.663548 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Updating router 4969b258-f606-4def-b666-09764c475749 port 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 binding host dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 20 04:24:45.711998 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 83f0181b-202d-4c5d-ab40-d746ebe28872 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:45.712488 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:45.789253 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:45.793241 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d62cd0f2-cd90-4957-b28f-b94f9f5824db req-2d7621dc-431e-48d7-a008-a2ff4a99a383 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 210 time: 0.0602088 May 20 04:24:45.793727 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-65344260-99ea-419c-9f68-fed55e74a4c4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:45.804209 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-65344260-99ea-419c-9f68-fed55e74a4c4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:45.821049 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-65344260-99ea-419c-9f68-fed55e74a4c4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Made reservation on behalf of 198bc8ab083f4a729f32b8ef27a6ee77 for: {'port': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:24:45.824657 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-65344260-99ea-419c-9f68-fed55e74a4c4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:45.955045 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6d61c197-7561-435f-be4c-f4b5ad5da5a5 req-c7de3be5-054c-4c33-861d-da6746bc4c0d tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:45.963105 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6d61c197-7561-435f-be4c-f4b5ad5da5a5 req-c7de3be5-054c-4c33-861d-da6746bc4c0d tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1317444 May 20 04:24:46.060696 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6d61c197-7561-435f-be4c-f4b5ad5da5a5 req-5b06c809-5716-4c31-8543-db0a6dd7bb79 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0963297 May 20 04:24:46.109245 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:46.266603 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:46.279132 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port 83f0181b-202d-4c5d-ab40-d746ebe28872, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=83f0181b-202d-4c5d-ab40-d746ebe28872,segment=NetworkSegment(3bd4c975-0dc6-43e6-b17a-8e2fa22a9507),segment_id=3bd4c975-0dc6-43e6-b17a-8e2fa22a9507)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:24:46.297499 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:46.297499 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 8f9c794c-3d7c-41d8-8927-b7ccdfe087ae up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:24:46.302968 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-94f56049-a48b-4a7d-baf4-11f0e75549ab req-aeef7465-b08b-4609-9609-84739d66ed12 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:46.304816 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-94f56049-a48b-4a7d-baf4-11f0e75549ab req-aeef7465-b08b-4609-9609-84739d66ed12 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 1098 time: 0.1045296 May 20 04:24:46.350468 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:46.376188 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-94f56049-a48b-4a7d-baf4-11f0e75549ab req-acb6456a-99e8-47a5-ae6c-417b07b3c1e2 service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0638652 May 20 04:24:46.400055 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-8d3e6e75-4f6c-4588-ae24-f5d8e7b29d26 None None] Provisioning complete for port 83f0181b-202d-4c5d-ab40-d746ebe28872 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:46.400439 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-8d3e6e75-4f6c-4588-ae24-f5d8e7b29d26 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:46.402958 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:24:46.414327 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Sync routers for ids ['5faa3599-06bc-45d7-8b3f-e21d104b087a'] in dv-cinder-782405-9-iscsi {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:24:46.424452 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Routers {'5faa3599-06bc-45d7-8b3f-e21d104b087a'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:24:46.425075 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:24:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5faa3599-06bc-45d7-8b3f-e21d104b087a'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:46.453167 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-65344260-99ea-419c-9f68-fed55e74a4c4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Allocated IP 172.20.0.3 (49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d/4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3/2cf59c2c-723c-4450-8b98-0809b16290a8) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:24:46.463743 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Notify callbacks [] for router, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:46.464985 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] 10.222.0.39 "POST /v2.0/routers HTTP/1.1" status: 201 len: 924 time: 4.0275526 May 20 04:24:46.468418 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 37174) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:46.540434 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=83f0181b-202d-4c5d-ab40-d746ebe28872,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:46.549973 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.200s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:46.667248 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 8f9c794c-3d7c-41d8-8927-b7ccdfe087ae triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:46.667561 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:46.703748 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-65344260-99ea-419c-9f68-fed55e74a4c4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:46.723651 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-65344260-99ea-419c-9f68-fed55e74a4c4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Transition to ACTIVE for port object 2cf59c2c-723c-4450-8b98-0809b16290a8 will not be triggered until provisioned by entity DHCP. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:24:46.729591 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Notify callbacks [] for port, precommit_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:46.755008 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-65344260-99ea-419c-9f68-fed55e74a4c4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:46.763954 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:46.775539 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Transition to ACTIVE for port object 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:24:46.784025 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Set router 5faa3599-06bc-45d7-8b3f-e21d104b087a gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:24:46.810849 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:46.824085 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-084677b6-59f9-4adc-ba2b-c0076081cb2e None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:46.846973 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-65344260-99ea-419c-9f68-fed55e74a4c4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Pushing event updated for resources: {'Port': ['ID=2cf59c2c-723c-4450-8b98-0809b16290a8,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:46.852253 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-65344260-99ea-419c-9f68-fed55e74a4c4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.088s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:46.863508 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-65344260-99ea-419c-9f68-fed55e74a4c4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Network 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:24:46.865068 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-65344260-99ea-419c-9f68-fed55e74a4c4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:24:46.868839 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-65344260-99ea-419c-9f68-fed55e74a4c4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:24:46.984712 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-65344260-99ea-419c-9f68-fed55e74a4c4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:46.985375 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-65344260-99ea-419c-9f68-fed55e74a4c4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:47.013403 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4e30e0a2-5f5b-4699-a542-3376f690db7e None None] Pushing event updated for resources: {'Port': ['ID=8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:47.019023 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 7dc69379-dd61-4add-8e0a-4ef00a211a0a up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:24:47.034067 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-65344260-99ea-419c-9f68-fed55e74a4c4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.4565027 May 20 04:24:47.050310 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-4e30e0a2-5f5b-4699-a542-3376f690db7e None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.226s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:47.060137 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Request body: {'port': {'device_id': '9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:47.322249 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Attempting to bind port 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:24:47.322820 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Attempting to bind port 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:24:47.323094 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Attempting to bind port 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 on network 14c537db-01aa-4360-a348-47f94c75e03b {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:24:47.329102 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-3dc22267-e7be-4bd4-8276-bd8ddc708a44 None None] Provisioning for port 2cf59c2c-723c-4450-8b98-0809b16290a8 completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:24:47.329873 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:47.332812 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 7dc69379-dd61-4add-8e0a-4ef00a211a0a triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:47.333190 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:47.360578 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 24, 45), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:24:47.360967 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Checking segment: {'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:24:47.361363 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:47.362866 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Bound using segment: {'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:24:47.363452 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Bound port: 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:24:47.363982 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:47.387524 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5faa3599-06bc-45d7-8b3f-e21d104b087a', 'name': 'tempest-TestVolumeBootPattern-1346048449-router', 'tenant_id': '207fc367830344daad06243fb26256f7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.182'}], 'enable_snat': True}, 'gw_port_id': '0e5ce97c-8370-428d-b569-67d6ffe03e3b', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:42Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 3, 'project_id': '207fc367830344daad06243fb26256f7', 'gw_port': {'id': '0e5ce97c-8370-428d-b569-67d6ffe03e3b', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:76:ec:16', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5faa3599-06bc-45d7-8b3f-e21d104b087a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 326, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.182', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:43Z', 'updated_at': '2021-05-20T04:24:43Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:47.392553 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Checking router: 5faa3599-06bc-45d7-8b3f-e21d104b087a for host: dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:24:47.392894 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Updating router 5faa3599-06bc-45d7-8b3f-e21d104b087a port 0e5ce97c-8370-428d-b569-67d6ffe03e3b binding host dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 20 04:24:47.394493 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-3dc22267-e7be-4bd4-8276-bd8ddc708a44 None None] Provisioning complete for port 2cf59c2c-723c-4450-8b98-0809b16290a8 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:47.394873 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-3dc22267-e7be-4bd4-8276-bd8ddc708a44 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:47.462845 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:47.540890 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-3dc22267-e7be-4bd4-8276-bd8ddc708a44 None None] Port 2cf59c2c-723c-4450-8b98-0809b16290a8 cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:24:47.585302 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] For port 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:24:47.619437 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d4aa1915-bafa-4902-86a1-4227861fea52 req-dcc17b39-8e0c-4b34-815b-0998eff1ba8e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:47.620424 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] For port 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:24:47.621426 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d4aa1915-bafa-4902-86a1-4227861fea52 req-dcc17b39-8e0c-4b34-815b-0998eff1ba8e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 931 time: 0.5261850 May 20 04:24:47.629702 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:47.635416 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:47.686345 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eb859b9e-e196-426e-937e-fefad450aaa8 req-d6459a3f-2dfc-4ef6-aa9f-8c0d2e033869 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:47.708402 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-eb859b9e-e196-426e-937e-fefad450aaa8 req-d6459a3f-2dfc-4ef6-aa9f-8c0d2e033869 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.3724246 May 20 04:24:47.718043 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4e30e0a2-5f5b-4699-a542-3376f690db7e None None] Pushing event updated for resources: {'Port': ['ID=8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:47.720749 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Routers {'4969b258-f606-4def-b666-09764c475749'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:24:47.721392 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:24:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['4969b258-f606-4def-b666-09764c475749'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:47.794518 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-4e30e0a2-5f5b-4699-a542-3376f690db7e None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.159s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:47.892996 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 2cf59c2c-723c-4450-8b98-0809b16290a8 {{(pid=91644) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:24:47.897502 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 83f0181b-202d-4c5d-ab40-d746ebe28872 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:24:47.915626 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:47.934014 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Notify callbacks [] for port, precommit_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:47.957190 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:47.961821 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Transition to ACTIVE for port object 2cf59c2c-723c-4450-8b98-0809b16290a8 will not be triggered until provisioned by entity L2. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:24:47.962320 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.provisioning_blocks [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Transition to ACTIVE for port object 0e5ce97c-8370-428d-b569-67d6ffe03e3b will not be triggered until provisioned by entity L2. {{(pid=91646) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:24:47.970698 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:47.971820 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:47.976397 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:47.978035 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:48.035072 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:48.054958 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2cc26b17-69b9-4548-b317-01903c12b399 None None] Pushing event updated for resources: {'Port': ['ID=0e5ce97c-8370-428d-b569-67d6ffe03e3b,revision_number=2']} {{(pid=91646) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:48.059531 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-2cc26b17-69b9-4548-b317-01903c12b399 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:48.078355 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Attempting to bind port 0e5ce97c-8370-428d-b569-67d6ffe03e3b on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:24:48.078907 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Attempting to bind port 0e5ce97c-8370-428d-b569-67d6ffe03e3b by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}] {{(pid=91646) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:24:48.079418 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Attempting to bind port 0e5ce97c-8370-428d-b569-67d6ffe03e3b on network 712ce8c4-8feb-4719-bf13-e06992125cd9 {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:24:48.085782 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 24, 45), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:24:48.086272 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Checking segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91646) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:24:48.086761 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:48.087322 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Bound using segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:24:48.087889 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Bound port: 0e5ce97c-8370-428d-b569-67d6ffe03e3b, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}}] {{(pid=91646) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:24:48.088356 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:48.212272 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 83f0181b-202d-4c5d-ab40-d746ebe28872 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:48.212647 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:48.289670 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Allocated IP 172.20.0.1 (b5c87200-df2c-432b-8fc4-24090c6d30f3/2927e09a-9033-4dd7-a313-77a3b77cd585/aedd7af1-ba1a-486f-a289-b5f1d46d3999) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:24:48.310895 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.db [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] For port 0e5ce97c-8370-428d-b569-67d6ffe03e3b, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91646) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:24:48.325324 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.db [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] For port 0e5ce97c-8370-428d-b569-67d6ffe03e3b, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=0e5ce97c-8370-428d-b569-67d6ffe03e3b,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91646) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:24:48.334329 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:48.350365 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:48.398811 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Pushing event updated for resources: {'Port': ['ID=2cf59c2c-723c-4450-8b98-0809b16290a8,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:48.403871 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.428s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:48.433045 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] Set router 4969b258-f606-4def-b666-09764c475749 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:24:48.445262 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Attempting to bind port 2cf59c2c-723c-4450-8b98-0809b16290a8 on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:24:48.445598 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Attempting to bind port 2cf59c2c-723c-4450-8b98-0809b16290a8 by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': 'b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 808, 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d'}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:24:48.445920 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Attempting to bind port 2cf59c2c-723c-4450-8b98-0809b16290a8 on network 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:24:48.456878 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Port 2cf59c2c-723c-4450-8b98-0809b16290a8 on network 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:24:48.457088 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Attempting to bind port 2cf59c2c-723c-4450-8b98-0809b16290a8 on network 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:24:48.472091 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 24, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:24:48.472420 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Checking segment: {'id': 'b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 808, 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:24:48.472862 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Bound using segment: {'id': 'b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 808, 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d'} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:24:48.473539 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Bound port: 2cf59c2c-723c-4450-8b98-0809b16290a8, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 808, 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d'}}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:24:48.479142 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:48.492364 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:48.521298 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.provisioning_blocks [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Transition to ACTIVE for port object aedd7af1-ba1a-486f-a289-b5f1d46d3999 will not be triggered until provisioned by entity DHCP. {{(pid=91643) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:24:48.529739 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Routers {'5faa3599-06bc-45d7-8b3f-e21d104b087a'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:24:48.534168 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:24:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5faa3599-06bc-45d7-8b3f-e21d104b087a'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:48.549157 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d4aa1915-bafa-4902-86a1-4227861fea52 req-a52783e8-871d-49df-987f-2dc0cb89d660 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.9130757 May 20 04:24:48.558191 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-eb859b9e-e196-426e-937e-fefad450aaa8 req-f964a88d-b734-4f78-9ab8-78ce39f5ec3a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.8486056 May 20 04:24:48.566737 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:48.573173 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:48.642449 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Network b5c87200-df2c-432b-8fc4-24090c6d30f3 is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:24:48.646742 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:24:48.649764 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:24:48.743006 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-8003132'] for router_interface, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:48.774013 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] For port 2cf59c2c-723c-4450-8b98-0809b16290a8, host hv-cinder-78240, cleared binding levels {{(pid=91644) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:24:48.777938 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] For port 2cf59c2c-723c-4450-8b98-0809b16290a8, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=2cf59c2c-723c-4450-8b98-0809b16290a8,segment=NetworkSegment(b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7),segment_id=b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7)] {{(pid=91644) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:24:48.789849 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:48.800366 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Pushing event updated for resources: {'Port': ['ID=aedd7af1-ba1a-486f-a289-b5f1d46d3999,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:48.803427 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:48.815265 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.241s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:48.853157 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2cc26b17-69b9-4548-b317-01903c12b399 None None] Pushing event updated for resources: {'Port': ['ID=0e5ce97c-8370-428d-b569-67d6ffe03e3b,revision_number=3']} {{(pid=91646) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:48.864790 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-2cc26b17-69b9-4548-b317-01903c12b399 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.514s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:48.942061 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:48.942913 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:48.951374 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] 10.222.0.74 "PUT /v2.0/ports/2cf59c2c-723c-4450-8b98-0809b16290a8 HTTP/1.1" status: 200 len: 1093 time: 1.8969350 May 20 04:24:48.963986 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b', 'tag': '2cf59c2c-723c-4450-8b98-0809b16290a8'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:24:49.066121 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:49.127195 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b', 'tag': '2cf59c2c-723c-4450-8b98-0809b16290a8', 'status': 'completed', 'code': 200} May 20 04:24:49.137501 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] Set router 5faa3599-06bc-45d7-8b3f-e21d104b087a gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:24:49.142032 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Pushing event updated for resources: {'Port': ['ID=2cf59c2c-723c-4450-8b98-0809b16290a8,revision_number=3']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:49.156974 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.363s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:49.216174 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-34d6abb0-30ba-4db1-a8c8-b63e3dfa1dcc service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:49.217527 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c07458d7-c7eb-425d-87ba-050927a4cd13 None None] Provisioning for port aedd7af1-ba1a-486f-a289-b5f1d46d3999 completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:24:49.223765 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c07458d7-c7eb-425d-87ba-050927a4cd13 None None] Provisioning complete for port aedd7af1-ba1a-486f-a289-b5f1d46d3999 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:49.224151 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c07458d7-c7eb-425d-87ba-050927a4cd13 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:49.225180 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-34d6abb0-30ba-4db1-a8c8-b63e3dfa1dcc service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=198bc8ab083f4a729f32b8ef27a6ee77&device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 1096 time: 0.2130623 May 20 04:24:49.271960 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:47Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:49.279863 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c5fb2015-54d8-4f68-b0fc-daf2e586c001 None None] The sync data for ids ['4969b258-f606-4def-b666-09764c475749'] in dv-cinder-782405-9-iscsi is: [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:47Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:24:49.293184 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c07458d7-c7eb-425d-87ba-050927a4cd13 None None] Port aedd7af1-ba1a-486f-a289-b5f1d46d3999 cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:24:49.351609 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-9b5d595e-70f8-4008-8619-fcb1447d6b3f service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=2cf59c2c-723c-4450-8b98-0809b16290a8 HTTP/1.1" status: 200 len: 217 time: 0.1144164 May 20 04:24:49.448468 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Notify callbacks [] for port, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:49.455102 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:49.525419 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:49.572659 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-7924897', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-8003163'] for router_interface, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:49.639168 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-198297f6-4533-4f75-b2ec-1cdd07bd1df7 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:49.640633 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-198297f6-4533-4f75-b2ec-1cdd07bd1df7 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3 HTTP/1.1" status: 200 len: 848 time: 0.2793109 May 20 04:24:49.642471 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5faa3599-06bc-45d7-8b3f-e21d104b087a', 'name': 'tempest-TestVolumeBootPattern-1346048449-router', 'tenant_id': '207fc367830344daad06243fb26256f7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.182'}], 'enable_snat': True}, 'gw_port_id': '0e5ce97c-8370-428d-b569-67d6ffe03e3b', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:42Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 3, 'project_id': '207fc367830344daad06243fb26256f7', 'gw_port': {'id': '0e5ce97c-8370-428d-b569-67d6ffe03e3b', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:76:ec:16', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5faa3599-06bc-45d7-8b3f-e21d104b087a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 326, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.182', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:43Z', 'updated_at': '2021-05-20T04:24:48Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:49.650639 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-87a76a52-431e-44a9-962d-c5f26dec99c9 None None] The sync data for ids ['5faa3599-06bc-45d7-8b3f-e21d104b087a'] in dv-cinder-782405-9-iscsi is: [{'id': '5faa3599-06bc-45d7-8b3f-e21d104b087a', 'name': 'tempest-TestVolumeBootPattern-1346048449-router', 'tenant_id': '207fc367830344daad06243fb26256f7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.182'}], 'enable_snat': True}, 'gw_port_id': '0e5ce97c-8370-428d-b569-67d6ffe03e3b', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:42Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 3, 'project_id': '207fc367830344daad06243fb26256f7', 'gw_port': {'id': '0e5ce97c-8370-428d-b569-67d6ffe03e3b', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:76:ec:16', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5faa3599-06bc-45d7-8b3f-e21d104b087a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 326, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.182', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:43Z', 'updated_at': '2021-05-20T04:24:48Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}] {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:24:49.769079 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-00a14f42-b44d-4829-9101-3ce1afe9d96e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:49.770944 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-00a14f42-b44d-4829-9101-3ce1afe9d96e service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1201296 May 20 04:24:49.876892 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:24:49.938883 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Pushing event updated for resources: {'Port': ['ID=aedd7af1-ba1a-486f-a289-b5f1d46d3999,revision_number=2']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:49.944402 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.418s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:49.972141 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-46dfe2bf-781e-40ed-bf98-aec60708de32 service neutron] 10.222.0.74 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1938934 May 20 04:24:49.974823 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:24:49.981992 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:49.982515 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:50.043314 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Network b5c87200-df2c-432b-8fc4-24090c6d30f3 is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:24:50.047082 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:24:50.187456 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ec7ea9e0-7f8b-415e-9197-af590462b96d req-74d68bae-ad2a-4a19-ad6c-e24680906cb0 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:50.197784 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ec7ea9e0-7f8b-415e-9197-af590462b96d req-74d68bae-ad2a-4a19-ad6c-e24680906cb0 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 943 time: 0.4451697 May 20 04:24:50.200406 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:50.232560 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0e25edb0-bd64-4d5f-b5eb-7db138e28f35 req-f844e7d2-3537-4196-a70a-c094d35789c5 service neutron] 10.222.0.74 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2537265 May 20 04:24:50.260593 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bb037b34-c70e-4e78-bdf6-32dab606399f req-0a548a68-1850-487c-885f-8f102a527272 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:50.271900 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bb037b34-c70e-4e78-bdf6-32dab606399f req-0a548a68-1850-487c-885f-8f102a527272 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.4179261 May 20 04:24:50.274419 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:24:50.293471 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:50.307438 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-4e30e0a2-5f5b-4699-a542-3376f690db7e None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.003s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:50.414862 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ec7ea9e0-7f8b-415e-9197-af590462b96d req-ccd9d1bd-9ffb-4bb6-958e-d45edb9438a4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.2144012 May 20 04:24:50.417260 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a17240ae-1b9b-467b-93f9-23111345a483 req-869aaaad-ac62-4437-9242-5f1f4dc862e9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:50.417495 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:50.418967 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-a17240ae-1b9b-467b-93f9-23111345a483 req-869aaaad-ac62-4437-9242-5f1f4dc862e9 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=198bc8ab083f4a729f32b8ef27a6ee77&device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 1096 time: 0.1039786 May 20 04:24:50.425344 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:50.425751 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:50.426163 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:50.426398 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:50.426729 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:50.427056 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:50.427416 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:50.427738 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:50.428078 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:50.428399 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:50.428715 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:50.429031 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:50.440326 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.133s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:50.457894 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-5e6ba503-80e0-416c-bc10-a02067213717 None None] Provisioning complete for port aedd7af1-ba1a-486f-a289-b5f1d46d3999 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:50.458613 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-5e6ba503-80e0-416c-bc10-a02067213717 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:50.562986 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-5e6ba503-80e0-416c-bc10-a02067213717 None None] Port aedd7af1-ba1a-486f-a289-b5f1d46d3999 cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:24:50.602996 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bb037b34-c70e-4e78-bdf6-32dab606399f req-9f237085-00db-4e75-b70d-5aee4696b583 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.3301349 May 20 04:24:50.670769 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Router 5faa3599-06bc-45d7-8b3f-e21d104b087a has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91643) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:24:50.699254 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a17240ae-1b9b-467b-93f9-23111345a483 req-383f2246-11cb-413f-aedc-732e1bf21026 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:50.702294 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-a17240ae-1b9b-467b-93f9-23111345a483 req-383f2246-11cb-413f-aedc-732e1bf21026 service neutron] 10.222.0.74 "GET /v2.0/networks?id=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d HTTP/1.1" status: 200 len: 910 time: 0.2713454 May 20 04:24:50.796977 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-a17240ae-1b9b-467b-93f9-23111345a483 req-cae8dcb3-3c44-4f05-b681-e19dc00f7e43 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=2cf59c2c-723c-4450-8b98-0809b16290a8 HTTP/1.1" status: 200 len: 217 time: 0.0859900 May 20 04:24:50.869632 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:24:50.870646 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a17240ae-1b9b-467b-93f9-23111345a483 req-3dcd5961-1604-4cf4-842e-9514b6da6cc2 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:50.872481 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-a17240ae-1b9b-467b-93f9-23111345a483 req-3dcd5961-1604-4cf4-842e-9514b6da6cc2 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3 HTTP/1.1" status: 200 len: 848 time: 0.0690646 May 20 04:24:50.883251 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] 10.222.0.39 "PUT /v2.0/routers/5faa3599-06bc-45d7-8b3f-e21d104b087a/add_router_interface HTTP/1.1" status: 200 len: 503 time: 4.4095225 May 20 04:24:50.978196 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a17240ae-1b9b-467b-93f9-23111345a483 req-a5224f99-8abf-4897-a3b3-263ca747ad10 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:50.979892 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-a17240ae-1b9b-467b-93f9-23111345a483 req-a5224f99-8abf-4897-a3b3-263ca747ad10 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0956614 May 20 04:24:51.168886 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d258e60-b483-45e6-b2b9-54715dc64f98 req-6b60b8f1-24a0-4a1d-9611-00965c772e83 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:51.170944 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1d258e60-b483-45e6-b2b9-54715dc64f98 req-6b60b8f1-24a0-4a1d-9611-00965c772e83 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=bec56d8703cf47f3b7193b7d1f55396d&device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 1098 time: 0.1828067 May 20 04:24:51.185504 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a52b001b-526b-44cc-a9e0-1b81c37ed351 req-60987334-38a1-4e09-9748-72997df73827 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:51.186928 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a52b001b-526b-44cc-a9e0-1b81c37ed351 req-60987334-38a1-4e09-9748-72997df73827 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1479077 May 20 04:24:51.198329 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-a17240ae-1b9b-467b-93f9-23111345a483 req-6877ebd2-b5f8-4b81-a5b0-242d48da7197 service neutron] 10.222.0.74 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2020316 May 20 04:24:51.321567 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a52b001b-526b-44cc-a9e0-1b81c37ed351 req-69dbaf16-eb57-4bb4-a201-692da51d4e5c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1142640 May 20 04:24:51.337494 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-a17240ae-1b9b-467b-93f9-23111345a483 req-8e97d0a7-f961-4a0a-98af-1541efb43e7e service neutron] 10.222.0.74 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1320283 May 20 04:24:51.348580 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d258e60-b483-45e6-b2b9-54715dc64f98 req-77566eb5-d537-444c-8127-330c8d4c2716 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:51.349532 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1d258e60-b483-45e6-b2b9-54715dc64f98 req-77566eb5-d537-444c-8127-330c8d4c2716 service neutron] 10.222.0.74 "GET /v2.0/networks?id=36b37213-8439-447d-9eba-a78baf7c698e HTTP/1.1" status: 200 len: 920 time: 0.1617229 May 20 04:24:51.429821 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1d258e60-b483-45e6-b2b9-54715dc64f98 req-0fa89198-8c79-4bfb-b7f0-ef482f84c689 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=706277c6-cd8c-4146-ab91-ed9504446690 HTTP/1.1" status: 200 len: 217 time: 0.0679033 May 20 04:24:51.493844 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d258e60-b483-45e6-b2b9-54715dc64f98 req-89c747ef-5876-403b-a1f9-24a4ca04ad2c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:51.495428 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1d258e60-b483-45e6-b2b9-54715dc64f98 req-89c747ef-5876-403b-a1f9-24a4ca04ad2c service neutron] 10.222.0.74 "GET /v2.0/subnets?id=b25ae9c8-e11d-45ee-bde2-332cc1538602 HTTP/1.1" status: 200 len: 859 time: 0.0589757 May 20 04:24:51.580348 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1d258e60-b483-45e6-b2b9-54715dc64f98 req-b2b575dd-fb94-430a-933c-ba2268b316c9 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=36b37213-8439-447d-9eba-a78baf7c698e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0777016 May 20 04:24:51.656873 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 37244) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:51.662409 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Request body: {'security_group': {'name': 'tempest-secgroup-smoke-195167850', 'description': 'tempest-secgroup-smoke-195167850 description', 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:51.670134 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:51.676468 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Usage tracker for resource:security_group and tenant:0e3baa0ea24d42f3ab59e40845b350fd is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:51.679766 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:51.687174 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:51.703040 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:51.729468 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-423baf20-569b-4b73-a4d4-73c19f113417 req-7babdae4-65e6-499f-adee-ee382b11cb0a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:51.736508 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-423baf20-569b-4b73-a4d4-73c19f113417 req-7babdae4-65e6-499f-adee-ee382b11cb0a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 943 time: 0.2047334 May 20 04:24:51.741995 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:51.748494 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Usage tracker for resource:security_group_rule and tenant:0e3baa0ea24d42f3ab59e40845b350fd is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:51.752565 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:51.759678 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:51.785503 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1d258e60-b483-45e6-b2b9-54715dc64f98 req-13f5b9cc-3ab5-4298-8c67-1bd370f554e9 service neutron] 10.222.0.74 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1969180 May 20 04:24:51.796605 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-423baf20-569b-4b73-a4d4-73c19f113417 req-979b38b3-cfe9-4eac-adf1-262bf1b377a9 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0591495 May 20 04:24:51.909334 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:24:51.912138 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:51.924118 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:51.941089 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-8b2b1b6d-5a79-44b8-b579-46d4407622d1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:51.968941 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:51.970162 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1792 time: 0.3123004 May 20 04:24:51.975132 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 37252) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:51.977017 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event updated for resources: {'SecurityGroup': ['ID=5a7943e2-185b-4112-9281-61ab71b1afeb,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:51.984783 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:51.985401 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:51.985401 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:51.985895 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:51.986246 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:51.986591 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:51.986936 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:51.987318 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:51.987648 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:51.987755 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-43079fb3-a87b-4e7b-8d07-d48d28838527 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Request body: {'security_group_rule': {'security_group_id': '5a7943e2-185b-4112-9281-61ab71b1afeb', 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:51.988048 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:51.988387 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:51.988722 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:51.992644 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.052s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:51.997453 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-43079fb3-a87b-4e7b-8d07-d48d28838527 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:52.004948 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-43079fb3-a87b-4e7b-8d07-d48d28838527 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Usage tracker for resource:security_group_rule and tenant:0e3baa0ea24d42f3ab59e40845b350fd is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:52.009301 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-43079fb3-a87b-4e7b-8d07-d48d28838527 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:52.012337 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:52.012712 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.022040 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-43079fb3-a87b-4e7b-8d07-d48d28838527 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:52.044705 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1d258e60-b483-45e6-b2b9-54715dc64f98 req-b4b68b93-e470-4011-aeaa-4792c28ecd07 service neutron] 10.222.0.74 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2470450 May 20 04:24:52.051191 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-43079fb3-a87b-4e7b-8d07-d48d28838527 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.113923 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-43079fb3-a87b-4e7b-8d07-d48d28838527 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.117849 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-43079fb3-a87b-4e7b-8d07-d48d28838527 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.120108 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9794c65b-e1a5-4b13-a066-c7e72f39e06a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:52.142454 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 0e5ce97c-8370-428d-b569-67d6ffe03e3b up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:24:52.145547 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-43079fb3-a87b-4e7b-8d07-d48d28838527 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=723e3b98-5f1f-42c1-baee-d2068e5be615,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:52.148098 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-43079fb3-a87b-4e7b-8d07-d48d28838527 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:52.155535 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-43079fb3-a87b-4e7b-8d07-d48d28838527 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.156612 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-43079fb3-a87b-4e7b-8d07-d48d28838527 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.1807580 May 20 04:24:52.159544 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 37256) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:52.164630 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-6dd45789-014e-42d0-b980-b872bc489607 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Request body: {'security_group_rule': {'security_group_id': '5a7943e2-185b-4112-9281-61ab71b1afeb', 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'egress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:52.170355 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-6dd45789-014e-42d0-b980-b872bc489607 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:52.174240 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-6dd45789-014e-42d0-b980-b872bc489607 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Usage tracker for resource:security_group_rule and tenant:0e3baa0ea24d42f3ab59e40845b350fd is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:52.177224 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-6dd45789-014e-42d0-b980-b872bc489607 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:52.183113 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-6dd45789-014e-42d0-b980-b872bc489607 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:52.199605 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6dd45789-014e-42d0-b980-b872bc489607 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.219921 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port 0e5ce97c-8370-428d-b569-67d6ffe03e3b completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:24:52.228384 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 0e5ce97c-8370-428d-b569-67d6ffe03e3b triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:52.229016 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.276986 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6dd45789-014e-42d0-b980-b872bc489607 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.281658 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6dd45789-014e-42d0-b980-b872bc489607 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.283961 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:52.309555 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6dd45789-014e-42d0-b980-b872bc489607 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=28bf063f-ef61-4bb4-a134-48485121b111,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:52.312274 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-6dd45789-014e-42d0-b980-b872bc489607 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:52.315625 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6dd45789-014e-42d0-b980-b872bc489607 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.319430 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-6dd45789-014e-42d0-b980-b872bc489607 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 763 time: 0.1563885 May 20 04:24:52.319644 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 37262) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:52.324642 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-91a6ea50-f0ab-45f2-ae0c-8fdb223178c7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Request body: {'security_group_rule': {'security_group_id': '5a7943e2-185b-4112-9281-61ab71b1afeb', 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'protocol': 'icmp', 'direction': 'ingress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:52.330193 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-91a6ea50-f0ab-45f2-ae0c-8fdb223178c7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:52.334064 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-91a6ea50-f0ab-45f2-ae0c-8fdb223178c7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Usage tracker for resource:security_group_rule and tenant:0e3baa0ea24d42f3ab59e40845b350fd is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:52.336883 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-91a6ea50-f0ab-45f2-ae0c-8fdb223178c7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Quota usage for security_group_rule was recalculated. Used quota:8. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:52.342764 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-91a6ea50-f0ab-45f2-ae0c-8fdb223178c7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 8; quota limit: 100; headroom:92 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:52.358673 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.363634 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-91a6ea50-f0ab-45f2-ae0c-8fdb223178c7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.374742 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port 0e5ce97c-8370-428d-b569-67d6ffe03e3b, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=0e5ce97c-8370-428d-b569-67d6ffe03e3b,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:24:52.379026 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.380964 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:52.433391 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-91a6ea50-f0ab-45f2-ae0c-8fdb223178c7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.437481 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-91a6ea50-f0ab-45f2-ae0c-8fdb223178c7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.447300 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=0e5ce97c-8370-428d-b569-67d6ffe03e3b,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:52.450348 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:52.455799 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-43079fb3-a87b-4e7b-8d07-d48d28838527 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:52.510679 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-91a6ea50-f0ab-45f2-ae0c-8fdb223178c7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=c41b0268-ea4a-4058-b475-150853ce5793,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:52.513132 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-91a6ea50-f0ab-45f2-ae0c-8fdb223178c7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.057s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:52.518784 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-91a6ea50-f0ab-45f2-ae0c-8fdb223178c7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.520169 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-91a6ea50-f0ab-45f2-ae0c-8fdb223178c7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.1996567 May 20 04:24:52.523162 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 37268) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:52.528902 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-dae9e12b-40d6-4925-bc03-edf01ea3b3cb tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Request body: {'security_group_rule': {'security_group_id': '5a7943e2-185b-4112-9281-61ab71b1afeb', 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'protocol': 'icmp', 'direction': 'egress'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:52.535268 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-dae9e12b-40d6-4925-bc03-edf01ea3b3cb tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:52.539406 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-dae9e12b-40d6-4925-bc03-edf01ea3b3cb tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Usage tracker for resource:security_group_rule and tenant:0e3baa0ea24d42f3ab59e40845b350fd is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:52.542518 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-dae9e12b-40d6-4925-bc03-edf01ea3b3cb tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Quota usage for security_group_rule was recalculated. Used quota:9. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:52.552713 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-dae9e12b-40d6-4925-bc03-edf01ea3b3cb tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 9; quota limit: 100; headroom:91 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:52.560209 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2c9c114e-c38d-4998-97cf-4e06498d64ff req-f938e39c-1e64-466a-8501-362da132150c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:52.562004 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2c9c114e-c38d-4998-97cf-4e06498d64ff req-f938e39c-1e64-466a-8501-362da132150c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1071954 May 20 04:24:52.570217 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dae9e12b-40d6-4925-bc03-edf01ea3b3cb tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.607664 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2c9c114e-c38d-4998-97cf-4e06498d64ff req-69dc21b2-95d8-4e8f-af0b-c357b17bfda6 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0397332 May 20 04:24:52.645012 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dae9e12b-40d6-4925-bc03-edf01ea3b3cb tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.649309 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dae9e12b-40d6-4925-bc03-edf01ea3b3cb tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.656610 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:52.678199 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dae9e12b-40d6-4925-bc03-edf01ea3b3cb tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=98ddf0fb-43f2-4005-b8ce-300b44bc4694,revision_number=0']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:52.681186 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-dae9e12b-40d6-4925-bc03-edf01ea3b3cb tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.025s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:52.687521 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dae9e12b-40d6-4925-bc03-edf01ea3b3cb tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.689541 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-dae9e12b-40d6-4925-bc03-edf01ea3b3cb tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 768 time: 0.1648700 May 20 04:24:52.694551 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 37272) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:52.703717 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Request body: {'security_group_rule': {'security_group_id': '5a7943e2-185b-4112-9281-61ab71b1afeb', 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'ingress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:52.711041 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:52.715098 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Usage tracker for resource:security_group_rule and tenant:0e3baa0ea24d42f3ab59e40845b350fd is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:52.718454 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Quota usage for security_group_rule was recalculated. Used quota:10. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:52.725143 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 10; quota limit: 100; headroom:90 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:52.738230 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.securitygroups_db [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Project 0e3baa0ea24d42f3ab59e40845b350fd added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 20 04:24:52.744300 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.810985 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.815549 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.817503 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:52.843632 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=0ccc5e81-65e9-451f-8345-9dd5d024f5e4,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:52.852110 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:52.852475 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:52.852824 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:52.853160 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:52.853523 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:52.853854 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:52.854199 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:52.854541 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:52.854901 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:52.855251 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:52.855595 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:52.855939 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:24:52.858746 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.861718 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 774 time: 0.1635020 May 20 04:24:52.861718 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:52.863048 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 37278) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:24:52.869398 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-d5a4391a-58b7-4042-a1e4-2e5cbaba65aa tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Request body: {'security_group_rule': {'security_group_id': '5a7943e2-185b-4112-9281-61ab71b1afeb', 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'egress'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:24:52.875828 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-d5a4391a-58b7-4042-a1e4-2e5cbaba65aa tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:24:52.881166 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-d5a4391a-58b7-4042-a1e4-2e5cbaba65aa tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Usage tracker for resource:security_group_rule and tenant:0e3baa0ea24d42f3ab59e40845b350fd is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:24:52.884265 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-d5a4391a-58b7-4042-a1e4-2e5cbaba65aa tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Quota usage for security_group_rule was recalculated. Used quota:11. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:24:52.890348 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-d5a4391a-58b7-4042-a1e4-2e5cbaba65aa tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 11; quota limit: 100; headroom:89 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:24:52.900824 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.db.securitygroups_db [None req-d5a4391a-58b7-4042-a1e4-2e5cbaba65aa tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Project 0e3baa0ea24d42f3ab59e40845b350fd added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 20 04:24:52.906712 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d5a4391a-58b7-4042-a1e4-2e5cbaba65aa tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.976681 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d5a4391a-58b7-4042-a1e4-2e5cbaba65aa tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.981398 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d5a4391a-58b7-4042-a1e4-2e5cbaba65aa tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:52.983821 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:53.008809 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d5a4391a-58b7-4042-a1e4-2e5cbaba65aa tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=2d442f84-8b34-4ba5-a194-de6ef45cdbb1,revision_number=0']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:53.011319 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d5a4391a-58b7-4042-a1e4-2e5cbaba65aa tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:53.020754 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d5a4391a-58b7-4042-a1e4-2e5cbaba65aa tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:53.023888 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-d5a4391a-58b7-4042-a1e4-2e5cbaba65aa tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 773 time: 0.1594961 May 20 04:24:53.101630 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fe70ff54-2380-489b-a4ea-2f3a9dec4d6a req-d6e8f743-21fb-4905-b362-a723b57d97d0 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:53.103066 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fe70ff54-2380-489b-a4ea-2f3a9dec4d6a req-d6e8f743-21fb-4905-b362-a723b57d97d0 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 943 time: 0.1806819 May 20 04:24:53.151565 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fe70ff54-2380-489b-a4ea-2f3a9dec4d6a req-e34bafcc-f5f3-407f-9f7e-98f28b6e770e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0411057 May 20 04:24:53.208012 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Sync routers for ids ['5faa3599-06bc-45d7-8b3f-e21d104b087a'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:24:53.218132 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Routers {'5faa3599-06bc-45d7-8b3f-e21d104b087a'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:24:53.218754 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:24:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5faa3599-06bc-45d7-8b3f-e21d104b087a'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:53.438849 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Set router 5faa3599-06bc-45d7-8b3f-e21d104b087a gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:24:53.755761 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 0e5ce97c-8370-428d-b569-67d6ffe03e3b up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:24:53.819004 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7e979dbd-1010-44bd-a2fe-145abca23de9 req-a71534d4-f0ae-4eb6-aa5f-eaf2a58de850 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:53.819666 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5faa3599-06bc-45d7-8b3f-e21d104b087a', 'name': 'tempest-TestVolumeBootPattern-1346048449-router', 'tenant_id': '207fc367830344daad06243fb26256f7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.182'}], 'enable_snat': True}, 'gw_port_id': '0e5ce97c-8370-428d-b569-67d6ffe03e3b', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:42Z', 'updated_at': '2021-05-20T04:24:48Z', 'revision_number': 4, 'project_id': '207fc367830344daad06243fb26256f7', 'gw_port': {'id': '0e5ce97c-8370-428d-b569-67d6ffe03e3b', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:76:ec:16', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5faa3599-06bc-45d7-8b3f-e21d104b087a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 326, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.182', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:43Z', 'updated_at': '2021-05-20T04:24:52Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'aedd7af1-ba1a-486f-a289-b5f1d46d3999', 'name': '', 'network_id': 'b5c87200-df2c-432b-8fc4-24090c6d30f3', 'tenant_id': '207fc367830344daad06243fb26256f7', 'mac_address': 'fa:16:3e:79:a2:98', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5faa3599-06bc-45d7-8b3f-e21d104b087a', 'device_owner': 'network:router_interface', 'standard_attr_id': 328, 'fixed_ips': [{'subnet_id': '2927e09a-9033-4dd7-a313-77a3b77cd585', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:48Z', 'updated_at': '2021-05-20T04:24:49Z', 'revision_number': 2, 'project_id': '207fc367830344daad06243fb26256f7', 'subnets': [{'id': '2927e09a-9033-4dd7-a313-77a3b77cd585', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:53.820521 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7e979dbd-1010-44bd-a2fe-145abca23de9 req-a71534d4-f0ae-4eb6-aa5f-eaf2a58de850 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0863037 May 20 04:24:53.825808 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Checking router: 5faa3599-06bc-45d7-8b3f-e21d104b087a for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:24:53.826166 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Updating router 5faa3599-06bc-45d7-8b3f-e21d104b087a port aedd7af1-ba1a-486f-a289-b5f1d46d3999 binding host dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 20 04:24:53.866168 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7e979dbd-1010-44bd-a2fe-145abca23de9 req-acdee9f4-b73b-45fa-8d9d-9d54cac8cb78 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0397284 May 20 04:24:53.884426 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:53.898111 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 0e5ce97c-8370-428d-b569-67d6ffe03e3b triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:53.898483 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:54.318479 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Notify callbacks [] for port, precommit_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:54.329952 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Transition to ACTIVE for port object aedd7af1-ba1a-486f-a289-b5f1d46d3999 will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:24:54.337898 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:54.338732 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f8aedde5-bd8b-4fe6-a38e-58171e41b371 req-083dc5c1-a976-48a8-a781-d4828aa4f988 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:54.340329 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f8aedde5-bd8b-4fe6-a38e-58171e41b371 req-083dc5c1-a976-48a8-a781-d4828aa4f988 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 943 time: 0.0805283 May 20 04:24:54.341551 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-08b47d33-b480-4f94-917f-e18e021c1929 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:54.384921 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f8aedde5-bd8b-4fe6-a38e-58171e41b371 req-066bd270-d754-415d-b34a-4aa62a508f31 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0383942 May 20 04:24:54.529846 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ace8ac61-9ace-48af-93fb-1ea478ec7cb2 None None] Pushing event updated for resources: {'Port': ['ID=aedd7af1-ba1a-486f-a289-b5f1d46d3999,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:54.532796 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-ace8ac61-9ace-48af-93fb-1ea478ec7cb2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.191s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:54.553694 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Attempting to bind port aedd7af1-ba1a-486f-a289-b5f1d46d3999 on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:24:54.554068 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Attempting to bind port aedd7af1-ba1a-486f-a289-b5f1d46d3999 by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '3bd4c975-0dc6-43e6-b17a-8e2fa22a9507', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 810, 'network_id': 'b5c87200-df2c-432b-8fc4-24090c6d30f3'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:24:54.554435 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Attempting to bind port aedd7af1-ba1a-486f-a289-b5f1d46d3999 on network b5c87200-df2c-432b-8fc4-24090c6d30f3 {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:24:54.559894 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 24, 45), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:24:54.560272 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Checking segment: {'id': '3bd4c975-0dc6-43e6-b17a-8e2fa22a9507', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 810, 'network_id': 'b5c87200-df2c-432b-8fc4-24090c6d30f3'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:24:54.560676 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:54.561068 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Bound using segment: {'id': '3bd4c975-0dc6-43e6-b17a-8e2fa22a9507', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 810, 'network_id': 'b5c87200-df2c-432b-8fc4-24090c6d30f3'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:24:54.561532 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Bound port: aedd7af1-ba1a-486f-a289-b5f1d46d3999, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '3bd4c975-0dc6-43e6-b17a-8e2fa22a9507', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 810, 'network_id': 'b5c87200-df2c-432b-8fc4-24090c6d30f3'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:24:54.561906 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:54.657998 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] For port aedd7af1-ba1a-486f-a289-b5f1d46d3999, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:24:54.661882 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] For port aedd7af1-ba1a-486f-a289-b5f1d46d3999, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=aedd7af1-ba1a-486f-a289-b5f1d46d3999,segment=NetworkSegment(3bd4c975-0dc6-43e6-b17a-8e2fa22a9507),segment_id=3bd4c975-0dc6-43e6-b17a-8e2fa22a9507)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:24:54.667606 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:54.671283 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:54.746064 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Routers {'5faa3599-06bc-45d7-8b3f-e21d104b087a'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:24:54.746675 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:24:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5faa3599-06bc-45d7-8b3f-e21d104b087a'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:54.812662 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ace8ac61-9ace-48af-93fb-1ea478ec7cb2 None None] Pushing event updated for resources: {'Port': ['ID=aedd7af1-ba1a-486f-a289-b5f1d46d3999,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:54.817535 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-ace8ac61-9ace-48af-93fb-1ea478ec7cb2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.146s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:54.926683 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] Set router 5faa3599-06bc-45d7-8b3f-e21d104b087a gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:24:55.057850 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-155b5e50-f31d-4671-93ec-ce0eb428b397 req-a6bbf337-28cd-4b37-ac70-56e20a42732c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:55.059323 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-155b5e50-f31d-4671-93ec-ce0eb428b397 req-a6bbf337-28cd-4b37-ac70-56e20a42732c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0847814 May 20 04:24:55.108302 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-155b5e50-f31d-4671-93ec-ce0eb428b397 req-94c3f88a-b969-46a1-839c-3376ba98b0cd tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0428228 May 20 04:24:55.275032 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5faa3599-06bc-45d7-8b3f-e21d104b087a', 'name': 'tempest-TestVolumeBootPattern-1346048449-router', 'tenant_id': '207fc367830344daad06243fb26256f7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.182'}], 'enable_snat': True}, 'gw_port_id': '0e5ce97c-8370-428d-b569-67d6ffe03e3b', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:42Z', 'updated_at': '2021-05-20T04:24:48Z', 'revision_number': 4, 'project_id': '207fc367830344daad06243fb26256f7', 'gw_port': {'id': '0e5ce97c-8370-428d-b569-67d6ffe03e3b', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:76:ec:16', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5faa3599-06bc-45d7-8b3f-e21d104b087a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 326, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.182', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:43Z', 'updated_at': '2021-05-20T04:24:52Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'aedd7af1-ba1a-486f-a289-b5f1d46d3999', 'name': '', 'network_id': 'b5c87200-df2c-432b-8fc4-24090c6d30f3', 'tenant_id': '207fc367830344daad06243fb26256f7', 'mac_address': 'fa:16:3e:79:a2:98', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5faa3599-06bc-45d7-8b3f-e21d104b087a', 'device_owner': 'network:router_interface', 'standard_attr_id': 328, 'fixed_ips': [{'subnet_id': '2927e09a-9033-4dd7-a313-77a3b77cd585', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:48Z', 'updated_at': '2021-05-20T04:24:54Z', 'revision_number': 4, 'project_id': '207fc367830344daad06243fb26256f7', 'subnets': [{'id': '2927e09a-9033-4dd7-a313-77a3b77cd585', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:24:55.279179 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9f0b8c8f-dc44-49bd-9428-5dc21a81096f None None] The sync data for ids ['5faa3599-06bc-45d7-8b3f-e21d104b087a'] in dv-cinder-782405-9-iscsi is: [{'id': '5faa3599-06bc-45d7-8b3f-e21d104b087a', 'name': 'tempest-TestVolumeBootPattern-1346048449-router', 'tenant_id': '207fc367830344daad06243fb26256f7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.182'}], 'enable_snat': True}, 'gw_port_id': '0e5ce97c-8370-428d-b569-67d6ffe03e3b', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:42Z', 'updated_at': '2021-05-20T04:24:48Z', 'revision_number': 4, 'project_id': '207fc367830344daad06243fb26256f7', 'gw_port': {'id': '0e5ce97c-8370-428d-b569-67d6ffe03e3b', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:76:ec:16', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5faa3599-06bc-45d7-8b3f-e21d104b087a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 326, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.182', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:43Z', 'updated_at': '2021-05-20T04:24:52Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'aedd7af1-ba1a-486f-a289-b5f1d46d3999', 'name': '', 'network_id': 'b5c87200-df2c-432b-8fc4-24090c6d30f3', 'tenant_id': '207fc367830344daad06243fb26256f7', 'mac_address': 'fa:16:3e:79:a2:98', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5faa3599-06bc-45d7-8b3f-e21d104b087a', 'device_owner': 'network:router_interface', 'standard_attr_id': 328, 'fixed_ips': [{'subnet_id': '2927e09a-9033-4dd7-a313-77a3b77cd585', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:48Z', 'updated_at': '2021-05-20T04:24:54Z', 'revision_number': 4, 'project_id': '207fc367830344daad06243fb26256f7', 'subnets': [{'id': '2927e09a-9033-4dd7-a313-77a3b77cd585', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:24:55.582052 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2dfebdd0-0366-4b1d-bce6-815d813122cf req-0b7907a2-b677-49d8-b90e-4526393d682c tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:55.583620 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2dfebdd0-0366-4b1d-bce6-815d813122cf req-0b7907a2-b677-49d8-b90e-4526393d682c tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 943 time: 0.0871143 May 20 04:24:55.631139 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2dfebdd0-0366-4b1d-bce6-815d813122cf req-3291b159-c16a-4fce-bfff-a47cfc75fd1a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0417230 May 20 04:24:55.786245 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device aedd7af1-ba1a-486f-a289-b5f1d46d3999 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:24:55.864635 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port aedd7af1-ba1a-486f-a289-b5f1d46d3999 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:24:55.869731 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port aedd7af1-ba1a-486f-a289-b5f1d46d3999 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:55.870107 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:55.987146 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:56.021366 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port aedd7af1-ba1a-486f-a289-b5f1d46d3999, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=aedd7af1-ba1a-486f-a289-b5f1d46d3999,segment=NetworkSegment(3bd4c975-0dc6-43e6-b17a-8e2fa22a9507),segment_id=3bd4c975-0dc6-43e6-b17a-8e2fa22a9507)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:24:56.026384 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:56.028461 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:24:56.102846 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=aedd7af1-ba1a-486f-a289-b5f1d46d3999,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:24:56.106158 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:24:56.298622 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c0b9b47e-5e84-4d69-a7fa-14b94e47f83e req-b51fe896-67e7-4775-80d2-34785f10ec2e tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:56.300105 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c0b9b47e-5e84-4d69-a7fa-14b94e47f83e req-b51fe896-67e7-4775-80d2-34785f10ec2e tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0859139 May 20 04:24:56.344970 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c0b9b47e-5e84-4d69-a7fa-14b94e47f83e req-27b9f818-3267-4237-b6e4-066ec4d39196 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0389724 May 20 04:24:56.826602 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4fcfd122-db14-448f-9717-3b6b42c4ab8c req-99f3c478-401d-464c-96ff-300ef4f3f7e8 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:56.828076 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4fcfd122-db14-448f-9717-3b6b42c4ab8c req-99f3c478-401d-464c-96ff-300ef4f3f7e8 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 943 time: 0.0895562 May 20 04:24:56.873277 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4fcfd122-db14-448f-9717-3b6b42c4ab8c req-11ada58b-12a8-4f18-bdc4-600f6df36582 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0391481 May 20 04:24:57.551469 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1e31e935-26d3-4e55-9267-9f2e214da08b req-923cf125-50ac-418d-bff1-0d5fd102b74b tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:57.552900 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1e31e935-26d3-4e55-9267-9f2e214da08b req-923cf125-50ac-418d-bff1-0d5fd102b74b tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0888374 May 20 04:24:57.602962 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1e31e935-26d3-4e55-9267-9f2e214da08b req-bda00300-c82a-466d-8cf7-10f02ad05118 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0437591 May 20 04:24:57.776522 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device aedd7af1-ba1a-486f-a289-b5f1d46d3999 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:24:57.843184 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port aedd7af1-ba1a-486f-a289-b5f1d46d3999 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:24:57.843596 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:24:58.065340 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e9298703-295f-45ca-a353-690583ab7b36 req-a96c049e-1ea3-4b73-8711-334ff1f5a5bd tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:58.066840 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e9298703-295f-45ca-a353-690583ab7b36 req-a96c049e-1ea3-4b73-8711-334ff1f5a5bd tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 943 time: 0.0863576 May 20 04:24:58.115441 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e9298703-295f-45ca-a353-690583ab7b36 req-638bfff8-7324-4d06-9939-81db3f249ac0 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0426795 May 20 04:24:58.796342 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9ce46b6b-7f20-40c2-8968-52b3e1fb4e14 req-7d161bb6-3850-4255-ab82-2b2b37effc7e tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:58.797752 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9ce46b6b-7f20-40c2-8968-52b3e1fb4e14 req-7d161bb6-3850-4255-ab82-2b2b37effc7e tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0869536 May 20 04:24:58.844469 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9ce46b6b-7f20-40c2-8968-52b3e1fb4e14 req-3194a86d-5065-4450-90b3-810651f8b3c9 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0407219 May 20 04:24:59.314225 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-720b4b57-28c2-4a65-8fc5-55fae9d18f77 req-9a03601d-8ea1-4d1d-84cb-7d4c549296e1 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:24:59.315711 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-720b4b57-28c2-4a65-8fc5-55fae9d18f77 req-9a03601d-8ea1-4d1d-84cb-7d4c549296e1 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 943 time: 0.0898385 May 20 04:24:59.362921 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-720b4b57-28c2-4a65-8fc5-55fae9d18f77 req-f9508103-a60d-4e74-9132-ff3970f39b3a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0409064 May 20 04:25:00.040148 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-718314b9-5887-4af1-a0be-ac190b8a5af5 req-8900706c-5809-453d-b53f-2147d6decd5a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:00.041581 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-718314b9-5887-4af1-a0be-ac190b8a5af5 req-8900706c-5809-453d-b53f-2147d6decd5a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0862234 May 20 04:25:00.087192 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-718314b9-5887-4af1-a0be-ac190b8a5af5 req-1e81943d-2e8c-4fd4-a452-7df7ba4abb9b tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0397460 May 20 04:25:00.569976 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-579439bf-8dd9-424d-a144-a8d4a528d2c4 req-42059f72-0e1f-48b9-94fa-19fdc9efe513 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:00.571462 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-579439bf-8dd9-424d-a144-a8d4a528d2c4 req-42059f72-0e1f-48b9-94fa-19fdc9efe513 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 943 time: 0.0857599 May 20 04:25:00.621057 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-579439bf-8dd9-424d-a144-a8d4a528d2c4 req-988aecbf-e087-4273-9e21-89fddf082635 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0432231 May 20 04:25:00.662129 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 2cf59c2c-723c-4450-8b98-0809b16290a8 details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:25:00.799934 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 2cf59c2c-723c-4450-8b98-0809b16290a8, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=2cf59c2c-723c-4450-8b98-0809b16290a8,segment=NetworkSegment(b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7),segment_id=b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:25:00.815983 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': '2cf59c2c-723c-4450-8b98-0809b16290a8', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'port_id': '2cf59c2c-723c-4450-8b98-0809b16290a8', 'mac_address': 'fa:16:3e:b8:b0:10', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 808, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'ip_address': '172.20.0.3'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:25:00.876166 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:00.893617 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 2cf59c2c-723c-4450-8b98-0809b16290a8 {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:25:00.911644 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 2cf59c2c-723c-4450-8b98-0809b16290a8, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=2cf59c2c-723c-4450-8b98-0809b16290a8,segment=NetworkSegment(b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7),segment_id=b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:25:00.916432 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:00.918204 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-4e30e0a2-5f5b-4699-a542-3376f690db7e None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:00.982688 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=2cf59c2c-723c-4450-8b98-0809b16290a8,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:00.985754 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.067s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:01.290453 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8bd564e5-5b38-4d93-9d6a-56b1253ee737 req-e9a9a1ec-078b-46d6-a76d-7f85b63793b2 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:01.291946 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8bd564e5-5b38-4d93-9d6a-56b1253ee737 req-e9a9a1ec-078b-46d6-a76d-7f85b63793b2 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0883710 May 20 04:25:01.339773 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8bd564e5-5b38-4d93-9d6a-56b1253ee737 req-39fe94d8-2391-436e-8a80-01c636ba5459 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0415552 May 20 04:25:01.504579 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['2cf59c2c-723c-4450-8b98-0809b16290a8'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:25:01.947638 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-109fd96b-830a-4076-a9e0-90dee1c01856 req-15c352b1-21ca-4cbe-9773-fa65e3aad537 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:01.949148 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-109fd96b-830a-4076-a9e0-90dee1c01856 req-15c352b1-21ca-4cbe-9773-fa65e3aad537 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 944 time: 0.0994310 May 20 04:25:01.994232 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-109fd96b-830a-4076-a9e0-90dee1c01856 req-b2c8cefe-8c69-4304-9326-70382577c53c tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0394254 May 20 04:25:02.160430 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-e7864344-8717-42a0-90df-456a56133757 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:02.161891 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-e7864344-8717-42a0-90df-456a56133757 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/networks?id=54fc8d38-193b-49c4-b873-c5353723791f HTTP/1.1" status: 200 len: 815 time: 0.1431315 May 20 04:25:02.179825 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-fa973741-4803-4f2b-b8ca-85300fd70037 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/quotas/51fcbc78284541db832e4780e76158af HTTP/1.1" status: 200 len: 375 time: 0.0094013 May 20 04:25:02.281174 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-9233f75c-44ee-4a26-a994-6287fb2e1db7 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=51fcbc78284541db832e4780e76158af&fields=id HTTP/1.1" status: 200 len: 348 time: 0.0939233 May 20 04:25:02.479179 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-5069b3b1-2377-422b-9a1e-614816a645b6 service neutron] 10.222.0.39 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1910214 May 20 04:25:02.608935 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-a25b5f0f-5c0c-4127-9025-23f31cda64c7 service neutron] 10.222.0.39 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1238058 May 20 04:25:02.793668 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-80857e72-941b-47d2-abb0-bb768e33adde req-e53ad881-e9c1-4aa8-862d-395e58a240c5 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 210 time: 0.0484631 May 20 04:25:03.257403 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a42444c4-fafc-4a55-aad3-109f0f5da929 req-d4f3d694-74ff-412a-8432-cfc95670b9c7 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:03.258852 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a42444c4-fafc-4a55-aad3-109f0f5da929 req-d4f3d694-74ff-412a-8432-cfc95670b9c7 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1478388 May 20 04:25:03.266981 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-74ee8234-76a6-4f12-a4a1-1ec168b8b6cc req-d0516e61-6ff6-4e4c-87d9-27db34e70147 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:03.268372 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-74ee8234-76a6-4f12-a4a1-1ec168b8b6cc req-d0516e61-6ff6-4e4c-87d9-27db34e70147 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 944 time: 0.1144507 May 20 04:25:03.336411 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a42444c4-fafc-4a55-aad3-109f0f5da929 req-19c3efbe-b1ca-4634-b986-280bae74120a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0670917 May 20 04:25:03.340753 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-74ee8234-76a6-4f12-a4a1-1ec168b8b6cc req-76a30f49-45f6-473a-85e8-70125d11f9f1 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0588837 May 20 04:25:03.947836 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4d0f6bfb-c929-45be-9e6b-3374de5c5794 req-d4315189-12f2-4e97-a0b3-233c15c5048f tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 210 time: 0.0419304 May 20 04:25:04.251373 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-eff2ca70-751b-434c-891f-8b0d238aaa6c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:04.253235 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-eff2ca70-751b-434c-891f-8b0d238aaa6c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.74 "GET /v2.0/networks?id=54fc8d38-193b-49c4-b873-c5353723791f HTTP/1.1" status: 200 len: 815 time: 0.1572075 May 20 04:25:04.271330 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-81ac2e41-a193-4259-99da-80efe9fc4e50 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Request body: {'port': {'device_id': '2f093c2c-1922-4116-95f2-a2e99555016d', 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f', 'admin_state_up': True, 'tenant_id': '51fcbc78284541db832e4780e76158af'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:04.417884 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-81ac2e41-a193-4259-99da-80efe9fc4e50 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:04.427616 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-81ac2e41-a193-4259-99da-80efe9fc4e50 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:04.437342 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-81ac2e41-a193-4259-99da-80efe9fc4e50 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Made reservation on behalf of 51fcbc78284541db832e4780e76158af for: {'port': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:25:04.440911 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-81ac2e41-a193-4259-99da-80efe9fc4e50 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:04.652998 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2cf53878-84f5-448f-b6ef-d3adb135e88c req-de042b84-0b2d-4363-bfba-f83905646e28 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:04.654430 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2cf53878-84f5-448f-b6ef-d3adb135e88c req-de042b84-0b2d-4363-bfba-f83905646e28 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 944 time: 0.1903543 May 20 04:25:04.688453 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 2cf59c2c-723c-4450-8b98-0809b16290a8 up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:25:04.692754 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-81ac2e41-a193-4259-99da-80efe9fc4e50 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Allocated IP 172.20.0.4 (54fc8d38-193b-49c4-b873-c5353723791f/0f918085-8d33-4e36-a59e-7f270a2b1874/a859e036-1f75-40cc-b35b-68bdda10b71f) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:25:04.719087 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2cf53878-84f5-448f-b6ef-d3adb135e88c req-5a33de8c-a0fd-4c70-b9c3-df98d3a551a6 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0586758 May 20 04:25:04.769218 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port 2cf59c2c-723c-4450-8b98-0809b16290a8 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:25:04.774356 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port 2cf59c2c-723c-4450-8b98-0809b16290a8 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:04.774747 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:04.896874 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:04.925102 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-81ac2e41-a193-4259-99da-80efe9fc4e50 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:04.928202 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '2cf59c2c-723c-4450-8b98-0809b16290a8'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:25:04.941091 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 2cf59c2c-723c-4450-8b98-0809b16290a8, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=2cf59c2c-723c-4450-8b98-0809b16290a8,segment=NetworkSegment(b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7),segment_id=b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:25:04.948706 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:04.952166 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:04.957805 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-81ac2e41-a193-4259-99da-80efe9fc4e50 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:04.965660 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:05.000698 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['849868be-af51-4967-9a6c-93d23cecb450', '643e5696-1a48-46bc-9a75-8b39f176b597', 'b6bee055-19d1-4e07-bd15-16290eac96d9'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:25:05.039768 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '2cf59c2c-723c-4450-8b98-0809b16290a8', 'code': 200} May 20 04:25:05.097686 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=2cf59c2c-723c-4450-8b98-0809b16290a8,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:05.107894 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.155s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:05.149986 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-81ac2e41-a193-4259-99da-80efe9fc4e50 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Network 54fc8d38-193b-49c4-b873-c5353723791f is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:25:05.166992 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-81ac2e41-a193-4259-99da-80efe9fc4e50 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:25:05.183970 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-3c98127a-635a-4352-9c4c-486fa30a137b req-896c47df-b3e2-4d32-afaf-217a29ee8ba9 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.1260710 May 20 04:25:05.208792 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-81ac2e41-a193-4259-99da-80efe9fc4e50 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:25:05.259889 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-81ac2e41-a193-4259-99da-80efe9fc4e50 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Pushing event updated for resources: {'Port': ['ID=a859e036-1f75-40cc-b35b-68bdda10b71f,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:05.278531 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-81ac2e41-a193-4259-99da-80efe9fc4e50 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.311s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:05.308749 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-81ac2e41-a193-4259-99da-80efe9fc4e50 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:05.309380 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-81ac2e41-a193-4259-99da-80efe9fc4e50 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:05.345159 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-81ac2e41-a193-4259-99da-80efe9fc4e50 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.0820477 May 20 04:25:05.353746 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-96e88dc0-a5ac-4579-953e-1e5021f71c4c req-cd9baf9b-670b-4d31-8251-893bd463b42a tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:05.354440 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-96e88dc0-a5ac-4579-953e-1e5021f71c4c req-cd9baf9b-670b-4d31-8251-893bd463b42a tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Synchronizing usage tracker for tenant:51fcbc78284541db832e4780e76158af on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:25:05.361120 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Request body: {'port': {'device_id': '2f093c2c-1922-4116-95f2-a2e99555016d', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:05.458285 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-96e88dc0-a5ac-4579-953e-1e5021f71c4c req-cd9baf9b-670b-4d31-8251-893bd463b42a tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Unset dirty status for tenant:51fcbc78284541db832e4780e76158af on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:25:05.458706 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2914a760-18fe-40f7-b077-bebf31864997 req-13e7f333-b30b-458b-9a0d-ed83cefd10f2 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:05.460731 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-2914a760-18fe-40f7-b077-bebf31864997 req-13e7f333-b30b-458b-9a0d-ed83cefd10f2 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 1098 time: 0.1732609 May 20 04:25:05.465086 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-96e88dc0-a5ac-4579-953e-1e5021f71c4c req-cd9baf9b-670b-4d31-8251-893bd463b42a tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 931 time: 0.2933085 May 20 04:25:05.485524 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3c98127a-635a-4352-9c4c-486fa30a137b req-efad25db-937d-4890-82c2-c7dff937d274 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:05.487497 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-3c98127a-635a-4352-9c4c-486fa30a137b req-efad25db-937d-4890-82c2-c7dff937d274 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=51fcbc78284541db832e4780e76158af&device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 1099 time: 0.1987951 May 20 04:25:05.518324 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-2914a760-18fe-40f7-b077-bebf31864997 req-3b4e64a9-076a-4509-945c-8ffd9fd1325d service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0504448 May 20 04:25:05.547137 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:05.612022 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-96e88dc0-a5ac-4579-953e-1e5021f71c4c req-36a18795-3843-4a8b-b881-2737b8751419 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1460702 May 20 04:25:05.612845 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:05.759361 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3c98127a-635a-4352-9c4c-486fa30a137b req-b1ed626b-fbb8-4a9a-ae1a-78ae8e48aeb4 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:05.760818 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-3c98127a-635a-4352-9c4c-486fa30a137b req-b1ed626b-fbb8-4a9a-ae1a-78ae8e48aeb4 service neutron] 10.222.0.74 "GET /v2.0/networks?id=54fc8d38-193b-49c4-b873-c5353723791f HTTP/1.1" status: 200 len: 916 time: 0.2461238 May 20 04:25:05.902721 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-3c98127a-635a-4352-9c4c-486fa30a137b req-ee90d347-3a57-4b18-a702-1656aa8afa33 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.14&port_id=849868be-af51-4967-9a6c-93d23cecb450 HTTP/1.1" status: 200 len: 217 time: 0.0672288 May 20 04:25:05.914191 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 37540) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:05.971107 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3c98127a-635a-4352-9c4c-486fa30a137b req-27729798-747a-415b-8ca9-87f563f0ee66 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:05.972971 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-3c98127a-635a-4352-9c4c-486fa30a137b req-27729798-747a-415b-8ca9-87f563f0ee66 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=0f918085-8d33-4e36-a59e-7f270a2b1874 HTTP/1.1" status: 200 len: 855 time: 0.0629535 May 20 04:25:05.997711 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-01d1d42a-e46e-402a-8b34-fc5b9bb76a24 req-b4d18882-0e51-4c00-a4be-21570f6b7bcf tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:05.999242 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-01d1d42a-e46e-402a-8b34-fc5b9bb76a24 req-b4d18882-0e51-4c00-a4be-21570f6b7bcf tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.1447952 May 20 04:25:06.043100 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-3c98127a-635a-4352-9c4c-486fa30a137b req-ecd71472-5a59-4213-b012-a6200e262840 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=54fc8d38-193b-49c4-b873-c5353723791f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0615273 May 20 04:25:06.082979 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id a859e036-1f75-40cc-b35b-68bdda10b71f {{(pid=91643) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:25:06.107555 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Notify callbacks [] for port, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:06.117721 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-01d1d42a-e46e-402a-8b34-fc5b9bb76a24 req-1a3c6ea6-66ff-447a-bd9b-1da2d1e316af tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0892441 May 20 04:25:06.118513 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-762e6639-b8ea-4715-bcdf-12d9eaf7578c admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:06.143386 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-762e6639-b8ea-4715-bcdf-12d9eaf7578c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:06.145703 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.provisioning_blocks [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Transition to ACTIVE for port object a859e036-1f75-40cc-b35b-68bdda10b71f will not be triggered until provisioned by entity L2. {{(pid=91643) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:25:06.148275 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-762e6639-b8ea-4715-bcdf-12d9eaf7578c admin admin] Usage tracker for resource:security_group_rule and tenant:af7f98cdc34343fc89f05e76cb3fee9e is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:25:06.151179 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-762e6639-b8ea-4715-bcdf-12d9eaf7578c admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:25:06.158193 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:06.163388 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-762e6639-b8ea-4715-bcdf-12d9eaf7578c admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:06.165370 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-8032a9e3-ff70-434d-a3f1-275d35ca238c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:06.199109 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-3c98127a-635a-4352-9c4c-486fa30a137b req-1478a438-8fbb-493a-80e9-1d0f9045bd13 service neutron] 10.222.0.74 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1473219 May 20 04:25:06.314142 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Pushing event updated for resources: {'Port': ['ID=a859e036-1f75-40cc-b35b-68bdda10b71f,revision_number=2']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:06.342445 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.177s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:06.467119 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Attempting to bind port a859e036-1f75-40cc-b35b-68bdda10b71f on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:25:06.467469 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Attempting to bind port a859e036-1f75-40cc-b35b-68bdda10b71f by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': '6fe99325-a72c-48bf-abc7-9bbcc4b753ff', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 836, 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f'}] {{(pid=91643) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:25:06.467812 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Attempting to bind port a859e036-1f75-40cc-b35b-68bdda10b71f on network 54fc8d38-193b-49c4-b873-c5353723791f {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:25:06.474475 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-762e6639-b8ea-4715-bcdf-12d9eaf7578c admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:06.476908 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Port a859e036-1f75-40cc-b35b-68bdda10b71f on network 54fc8d38-193b-49c4-b873-c5353723791f not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:25:06.477242 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Attempting to bind port a859e036-1f75-40cc-b35b-68bdda10b71f on network 54fc8d38-193b-49c4-b873-c5353723791f {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:25:06.482820 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-762e6639-b8ea-4715-bcdf-12d9eaf7578c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:06.490826 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:06.498651 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 25, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:25:06.498991 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Checking segment: {'id': '6fe99325-a72c-48bf-abc7-9bbcc4b753ff', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 836, 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91643) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:25:06.499411 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Bound using segment: {'id': '6fe99325-a72c-48bf-abc7-9bbcc4b753ff', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 836, 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f'} {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:25:06.499866 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Bound port: a859e036-1f75-40cc-b35b-68bdda10b71f, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '6fe99325-a72c-48bf-abc7-9bbcc4b753ff', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 836, 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f'}}] {{(pid=91643) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:25:06.500215 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:06.519880 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-3c98127a-635a-4352-9c4c-486fa30a137b req-302f81b1-d145-4910-9621-4b497bc16f58 service neutron] 10.222.0.74 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1277514 May 20 04:25:06.579687 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-762e6639-b8ea-4715-bcdf-12d9eaf7578c admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=8c4f750e-4661-4f6e-8711-eb45485a96a8,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:06.589484 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-762e6639-b8ea-4715-bcdf-12d9eaf7578c admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=af7f98cdc34343fc89f05e76cb3fee9e&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6743042 May 20 04:25:06.590472 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-762e6639-b8ea-4715-bcdf-12d9eaf7578c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.100s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:06.592971 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 37550) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:06.631840 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] For port a859e036-1f75-40cc-b35b-68bdda10b71f, host hv-cinder-78240, cleared binding levels {{(pid=91643) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:25:06.634893 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] For port a859e036-1f75-40cc-b35b-68bdda10b71f, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=a859e036-1f75-40cc-b35b-68bdda10b71f,segment=NetworkSegment(6fe99325-a72c-48bf-abc7-9bbcc4b753ff),segment_id=6fe99325-a72c-48bf-abc7-9bbcc4b753ff)] {{(pid=91643) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:25:06.642743 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:06.648225 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:06.660740 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a67c8edf-4ad9-478b-a992-8284fc66b0d2 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:06.689494 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a67c8edf-4ad9-478b-a992-8284fc66b0d2 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:06.703820 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:06.704584 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:06.705952 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] 10.222.0.74 "PUT /v2.0/ports/a859e036-1f75-40cc-b35b-68bdda10b71f HTTP/1.1" status: 200 len: 1093 time: 1.3487754 May 20 04:25:06.709195 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '2f093c2c-1922-4116-95f2-a2e99555016d', 'tag': 'a859e036-1f75-40cc-b35b-68bdda10b71f'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:25:06.741018 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Pushing event updated for resources: {'Port': ['ID=a859e036-1f75-40cc-b35b-68bdda10b71f,revision_number=3']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:06.748538 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.100s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:06.770127 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a67c8edf-4ad9-478b-a992-8284fc66b0d2 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:06.774718 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-3123fe9b-e4b1-4ebb-867c-4be5a17ff978 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:06.786160 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a67c8edf-4ad9-478b-a992-8284fc66b0d2 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=8c4f750e-4661-4f6e-8711-eb45485a96a8,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:06.792363 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-a67c8edf-4ad9-478b-a992-8284fc66b0d2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:06.793155 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a67c8edf-4ad9-478b-a992-8284fc66b0d2 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:06.795266 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-a67c8edf-4ad9-478b-a992-8284fc66b0d2 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/8c4f750e-4661-4f6e-8711-eb45485a96a8 HTTP/1.1" status: 204 len: 168 time: 0.2002482 May 20 04:25:06.795605 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '2f093c2c-1922-4116-95f2-a2e99555016d', 'tag': 'a859e036-1f75-40cc-b35b-68bdda10b71f', 'status': 'completed', 'code': 200} May 20 04:25:06.906107 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-5b454983-85ad-482c-9736-20dfd2d39ba7 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:06.907589 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-5b454983-85ad-482c-9736-20dfd2d39ba7 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=51fcbc78284541db832e4780e76158af&device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 1096 time: 0.1002798 May 20 04:25:06.918785 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-728a1b65-36f3-4307-b7a8-4f89b4a03777 req-f0fad94a-fe48-4e80-8ddd-4e1c0ed8a089 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:06.920647 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-728a1b65-36f3-4307-b7a8-4f89b4a03777 req-f0fad94a-fe48-4e80-8ddd-4e1c0ed8a089 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 943 time: 0.1684911 May 20 04:25:06.999545 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-19fe9bee-1822-4c79-bc35-16598c052304 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=a859e036-1f75-40cc-b35b-68bdda10b71f HTTP/1.1" status: 200 len: 217 time: 0.0779443 May 20 04:25:07.060224 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-728a1b65-36f3-4307-b7a8-4f89b4a03777 req-9f01cae9-dd57-4cd3-99d7-d0f4f8c60d8d tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0831573 May 20 04:25:07.085958 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-f7a60882-40b3-4aae-bc05-419d6529522d service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:07.087302 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-f7a60882-40b3-4aae-bc05-419d6529522d service neutron] 10.222.0.74 "GET /v2.0/subnets?id=0f918085-8d33-4e36-a59e-7f270a2b1874 HTTP/1.1" status: 200 len: 855 time: 0.0727916 May 20 04:25:07.141950 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-6d913623-a9fc-47ab-822c-46b2dc65aff3 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=54fc8d38-193b-49c4-b873-c5353723791f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0461206 May 20 04:25:07.345451 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-971fe48d-bd93-4a53-8c6b-42859344a33d service neutron] 10.222.0.74 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1953561 May 20 04:25:07.535850 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fc3fa0c4-e435-412f-9456-a155c68b1182 req-a45bb200-0cbb-469a-943e-a67effa9089d tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:07.537288 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fc3fa0c4-e435-412f-9456-a155c68b1182 req-a45bb200-0cbb-469a-943e-a67effa9089d tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.2739801 May 20 04:25:07.597614 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 37586) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:07.612141 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fc3fa0c4-e435-412f-9456-a155c68b1182 req-9a679512-efcd-449a-841d-0c8f582d7769 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0682156 May 20 04:25:07.617863 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a654a3b3-777b-4e21-bc44-b1743a9b0b39 req-ab70a261-d186-491c-b28c-882c592f2d41 service neutron] 10.222.0.74 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2639270 May 20 04:25:07.712915 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Request body: {'network': {'name': 'tempest-TestShelveInstance-251001111-network', 'tenant_id': '39febf983bf24778a300f287dbeca829'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:07.718360 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:07.722722 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Usage tracker for resource:network and tenant:39febf983bf24778a300f287dbeca829 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:25:07.725581 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:25:07.732010 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:07.742087 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Made reservation on behalf of 39febf983bf24778a300f287dbeca829 for: {'network': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:25:07.742718 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4c729b4b-9e4d-428d-a83a-793dc6fa1079 req-4b790818-79f3-4205-9501-604c4b64a9d0 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:07.744237 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4c729b4b-9e4d-428d-a83a-793dc6fa1079 req-4b790818-79f3-4205-9501-604c4b64a9d0 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=51fcbc78284541db832e4780e76158af&device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 1096 time: 0.0741501 May 20 04:25:07.744464 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-585852'] for network, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:07.756895 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:07.771754 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:07.775914 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Usage tracker for resource:security_group_rule and tenant:39febf983bf24778a300f287dbeca829 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:25:07.778833 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:25:07.784714 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:07.800381 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:25:07.870786 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:07.875527 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:07.879916 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-a67c8edf-4ad9-478b-a992-8284fc66b0d2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:07.889383 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4c729b4b-9e4d-428d-a83a-793dc6fa1079 req-7bcf609b-477f-41d2-ae69-1395b0dbdb63 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:07.889383 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4c729b4b-9e4d-428d-a83a-793dc6fa1079 req-7bcf609b-477f-41d2-ae69-1395b0dbdb63 service neutron] 10.222.0.74 "GET /v2.0/networks?id=54fc8d38-193b-49c4-b873-c5353723791f HTTP/1.1" status: 200 len: 916 time: 0.1377299 May 20 04:25:07.914053 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 806} {{(pid=91644) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 20 04:25:07.922725 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=3e4fedd7-e290-4dd9-8e9b-c2652f05a367,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:07.937139 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.056s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:07.942097 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:07.942525 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.db.segments_db [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Added segment c7d77442-5ac6-48cf-ab59-3f5962fc1e3e of type vlan for network e57f6e29-4637-455e-af9d-0975c45c995c May 20 04:25:07.962993 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4c729b4b-9e4d-428d-a83a-793dc6fa1079 req-f98803de-7992-45cb-b41a-320a52bf15b6 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=a859e036-1f75-40cc-b35b-68bdda10b71f HTTP/1.1" status: 200 len: 217 time: 0.0657058 May 20 04:25:07.970318 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8760688516720'] for network, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:08.000122 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915'] for network, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:08.001408 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:08.001979 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:08.005281 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:08.032280 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4c729b4b-9e4d-428d-a83a-793dc6fa1079 req-7032d351-338f-44cb-99a0-cdc71eada7a2 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:08.033615 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4c729b4b-9e4d-428d-a83a-793dc6fa1079 req-7032d351-338f-44cb-99a0-cdc71eada7a2 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=0f918085-8d33-4e36-a59e-7f270a2b1874 HTTP/1.1" status: 200 len: 855 time: 0.0609841 May 20 04:25:08.082651 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4c729b4b-9e4d-428d-a83a-793dc6fa1079 req-27a618e4-d35d-462e-8404-3f61c68588e7 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=54fc8d38-193b-49c4-b873-c5353723791f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0406413 May 20 04:25:08.108202 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Pushing event updated for resources: {'Network': ['ID=e57f6e29-4637-455e-af9d-0975c45c995c,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:08.111168 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] 10.222.0.39 "POST /v2.0/networks HTTP/1.1" status: 201 len: 881 time: 0.5124702 May 20 04:25:08.114234 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.109s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:08.114721 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 37596) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:08.120228 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-59bf4400-3f5c-414b-9f5b-ed89e5c160e0 admin admin] Request body: {'subnet': {'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestShelveInstance-251001111-subnet', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'ip_version': 4}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:08.124935 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-59bf4400-3f5c-414b-9f5b-ed89e5c160e0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:08.125392 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Network e57f6e29-4637-455e-af9d-0975c45c995c details requested by agent ovs-agent-dv-cinder-782405-9-iscsi with host dv-cinder-782405-9-iscsi {{(pid=91645) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 20 04:25:08.128915 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-59bf4400-3f5c-414b-9f5b-ed89e5c160e0 admin admin] Usage tracker for resource:subnet and tenant:39febf983bf24778a300f287dbeca829 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:25:08.131858 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-59bf4400-3f5c-414b-9f5b-ed89e5c160e0 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:25:08.137581 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-59bf4400-3f5c-414b-9f5b-ed89e5c160e0 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:08.153845 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-59bf4400-3f5c-414b-9f5b-ed89e5c160e0 admin admin] Made reservation on behalf of 39febf983bf24778a300f287dbeca829 for: {'subnet': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:25:08.156408 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-59bf4400-3f5c-414b-9f5b-ed89e5c160e0 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-1810190'] for subnet, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:08.309207 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4c729b4b-9e4d-428d-a83a-793dc6fa1079 req-e2c74a0e-38c9-4e91-af1c-ed00b358b9c9 service neutron] 10.222.0.74 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2127657 May 20 04:25:08.328002 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-172f533d-cd13-4a67-9fe5-58191ce1dd86 req-25dc3c27-31f6-4374-9587-a312407bc43b tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:08.329459 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-172f533d-cd13-4a67-9fe5-58191ce1dd86 req-25dc3c27-31f6-4374-9587-a312407bc43b tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 943 time: 0.1205845 May 20 04:25:08.472414 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-172f533d-cd13-4a67-9fe5-58191ce1dd86 req-eb31aaca-6c28-4532-afda-d94731d6b498 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1350877 May 20 04:25:08.508050 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4c729b4b-9e4d-428d-a83a-793dc6fa1079 req-5e6318d1-51af-474a-a253-5b270cd989c8 service neutron] 10.222.0.74 "GET /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1531248 May 20 04:25:08.602682 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-59bf4400-3f5c-414b-9f5b-ed89e5c160e0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-1988978', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-1810236'] for subnet, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:08.607606 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:08.680428 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-59bf4400-3f5c-414b-9f5b-ed89e5c160e0 admin admin] Pushing event updated for resources: {'Subnet': ['ID=49523a89-260b-4732-abff-507d74fed4c4,revision_number=0']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:08.683718 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-59bf4400-3f5c-414b-9f5b-ed89e5c160e0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.076s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:08.689430 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-59bf4400-3f5c-414b-9f5b-ed89e5c160e0 admin admin] Checking segment: {'id': 'c7d77442-5ac6-48cf-ab59-3f5962fc1e3e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 806, 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:25:08.689876 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-59bf4400-3f5c-414b-9f5b-ed89e5c160e0 admin admin] Network hostable DHCP agents. Network: e57f6e29-4637-455e-af9d-0975c45c995c, hostable agents: ['fffae25a-f450-4182-8f9e-ac29d143f6c9'], hosted agents: [] {{(pid=91644) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 20 04:25:08.711186 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-59bf4400-3f5c-414b-9f5b-ed89e5c160e0 admin admin] Network e57f6e29-4637-455e-af9d-0975c45c995c is scheduled to be hosted by DHCP agent fffae25a-f450-4182-8f9e-ac29d143f6c9 with binding_index 1 {{(pid=91644) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 20 04:25:08.721229 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-59bf4400-3f5c-414b-9f5b-ed89e5c160e0 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-782405-9-iscsi, e57f6e29-4637-455e-af9d-0975c45c995c) {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:25:08.730203 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-59bf4400-3f5c-414b-9f5b-ed89e5c160e0 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:08.730778 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-59bf4400-3f5c-414b-9f5b-ed89e5c160e0 admin admin] Notify callbacks [] for subnet, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:08.731129 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-69728a9e-4600-4de5-9e89-105bc1c8c5df None None] Network e57f6e29-4637-455e-af9d-0975c45c995c requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:25:08.753673 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-59bf4400-3f5c-414b-9f5b-ed89e5c160e0 admin admin] 10.222.0.39 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 844 time: 0.6354842 May 20 04:25:08.754011 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 37606) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:08.759488 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Request body: {'router': {'name': 'tempest-TestShelveInstance-251001111-router', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:08.768840 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:08.775479 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Usage tracker for resource:router and tenant:39febf983bf24778a300f287dbeca829 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:25:08.778932 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:25:08.785622 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:08.798892 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-8071540', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-260712'] for router, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:08.800292 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-8056436', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-8053301', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-7881139', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-260707'] for router, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:08.815771 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5c11fbd-5a8c-4515-b364-d4390885703e req-88561273-ed65-4510-b1ed-c10c908717f7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:08.816183 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:08.817746 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b5c11fbd-5a8c-4515-b364-d4390885703e req-88561273-ed65-4510-b1ed-c10c908717f7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0901024 May 20 04:25:08.867401 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b5c11fbd-5a8c-4515-b364-d4390885703e req-dd397b2c-b797-47d8-8221-b57f87133457 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0442443 May 20 04:25:08.976478 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcp85c0b5c8-c799-5455-967d-3e9c2c8b4839-e57f6e29-4637-455e-af9d-0975c45c995c', 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'fixed_ips': [{'subnet_id': '49523a89-260b-4732-abff-507d74fed4c4'}]}} from dv-cinder-782405-9-iscsi. {{(pid=91645) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 20 04:25:08.977733 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:09.103159 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2955cc1f-a4ef-43fd-ae6d-df3cdd54b10b req-fe031168-c18c-4531-9b05-4e1d08a0bb50 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:09.105831 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2955cc1f-a4ef-43fd-ae6d-df3cdd54b10b req-fe031168-c18c-4531-9b05-4e1d08a0bb50 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.1077664 May 20 04:25:09.114653 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 37614) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:09.121823 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-0a7a16d5-9b20-4b9e-aec6-e1aa51febf3d tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Request body: {'floatingip': {'port_id': '2cf59c2c-723c-4450-8b98-0809b16290a8'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:09.156709 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.db_base_plugin_common [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Allocated IP 172.20.0.2 (e57f6e29-4637-455e-af9d-0975c45c995c/49523a89-260b-4732-abff-507d74fed4c4/a6b8dc33-0ff3-46bf-a09b-b512222443bb) {{(pid=91645) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:25:09.253601 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-8053308', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-8071430', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-260685'] for router, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:09.262062 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0a7a16d5-9b20-4b9e-aec6-e1aa51febf3d tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for floatingip, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:09.361756 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Notify callbacks [] for port, precommit_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:09.373095 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Transition to ACTIVE for port object a6b8dc33-0ff3-46bf-a09b-b512222443bb will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:25:09.395471 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Transition to ACTIVE for port object a6b8dc33-0ff3-46bf-a09b-b512222443bb will not be triggered until provisioned by entity DHCP. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:25:09.402878 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:09.406648 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:09.490251 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Network e57f6e29-4637-455e-af9d-0975c45c995c is already hosted by enough agents. {{(pid=91645) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:25:09.493167 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.base_scheduler [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Resources bound (agent type, host, resource id): {{(pid=91645) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:25:09.497029 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Received port after_create {{(pid=91645) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:25:09.576053 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Pushing event updated for resources: {'Port': ['ID=a6b8dc33-0ff3-46bf-a09b-b512222443bb,revision_number=1']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:09.579584 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.173s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:09.612162 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Attempting to bind port a6b8dc33-0ff3-46bf-a09b-b512222443bb on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:25:09.612583 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Attempting to bind port a6b8dc33-0ff3-46bf-a09b-b512222443bb by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': 'c7d77442-5ac6-48cf-ab59-3f5962fc1e3e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 806, 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:25:09.612941 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Attempting to bind port a6b8dc33-0ff3-46bf-a09b-b512222443bb on network e57f6e29-4637-455e-af9d-0975c45c995c {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:25:09.622231 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 24, 45), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:25:09.622608 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Checking segment: {'id': 'c7d77442-5ac6-48cf-ab59-3f5962fc1e3e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 806, 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:25:09.625735 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:09.625735 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Bound using segment: {'id': 'c7d77442-5ac6-48cf-ab59-3f5962fc1e3e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 806, 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:25:09.625735 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Bound port: a6b8dc33-0ff3-46bf-a09b-b512222443bb, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'c7d77442-5ac6-48cf-ab59-3f5962fc1e3e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 806, 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:25:09.625735 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:09.694934 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0ca3cfd-f980-4d03-82ef-82ef36f23aa2 req-926f058a-ec01-402b-8f6b-ef1fdd5ec573 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:09.702420 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b0ca3cfd-f980-4d03-82ef-82ef36f23aa2 req-926f058a-ec01-402b-8f6b-ef1fdd5ec573 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 943 time: 0.1038675 May 20 04:25:09.732286 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] For port a6b8dc33-0ff3-46bf-a09b-b512222443bb, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:25:09.736025 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] For port a6b8dc33-0ff3-46bf-a09b-b512222443bb, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=a6b8dc33-0ff3-46bf-a09b-b512222443bb,segment=NetworkSegment(c7d77442-5ac6-48cf-ab59-3f5962fc1e3e),segment_id=c7d77442-5ac6-48cf-ab59-3f5962fc1e3e)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:25:09.749367 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:09.756952 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:09.760201 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b0ca3cfd-f980-4d03-82ef-82ef36f23aa2 req-524ef09a-e1e9-45f0-915e-c2fc05b8ee31 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0568964 May 20 04:25:09.895119 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Pushing event updated for resources: {'Port': ['ID=a6b8dc33-0ff3-46bf-a09b-b512222443bb,revision_number=2']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:09.898298 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.141s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:10.149027 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:10.180519 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:10.403447 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0a7a16d5-9b20-4b9e-aec6-e1aa51febf3d tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for floatingip, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:10.412107 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0a7a16d5-9b20-4b9e-aec6-e1aa51febf3d tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-8003251'] for floatingip, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:10.587352 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Allocated IP 172.27.0.100 (712ce8c4-8feb-4719-bf13-e06992125cd9/e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5/37393b04-8a64-4737-953b-1c02d4efce9f) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:25:10.752827 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-5b2fba27-4415-4dfa-8de2-ec00b18d5e0f None None] Network e57f6e29-4637-455e-af9d-0975c45c995c requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:25:10.800131 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.db.l3_db [None req-0a7a16d5-9b20-4b9e-aec6-e1aa51febf3d tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Floating IP b1bce8cf-eac1-4b98-800e-1ec838d5f270 associated. External IP: 172.27.0.159, port: 2cf59c2c-723c-4450-8b98-0809b16290a8. May 20 04:25:10.882671 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c96f2de4-ee6e-4f67-8c6c-c9318545fd59 None None] Provisioning for port a6b8dc33-0ff3-46bf-a09b-b512222443bb completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:25:10.905885 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:11.015752 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5f9740b2-8f99-4633-8944-72cc103e5a25 req-febc3135-f1a7-4036-96f2-22baf4802dbd tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:11.018061 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5f9740b2-8f99-4633-8944-72cc103e5a25 req-febc3135-f1a7-4036-96f2-22baf4802dbd tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 943 time: 0.1380298 May 20 04:25:11.075733 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5f9740b2-8f99-4633-8944-72cc103e5a25 req-abcde7a3-47c6-4c11-8273-5d7fd4021c66 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0501959 May 20 04:25:11.130956 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:11.137059 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:11.212956 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:25:11.217854 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:25:11.223194 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:25:11.313672 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Pushing event updated for resources: {'Port': ['ID=37393b04-8a64-4737-953b-1c02d4efce9f,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:11.334307 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.197s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:11.482294 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-0a7a16d5-9b20-4b9e-aec6-e1aa51febf3d tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Router 10aeccef-e762-4811-a411-a1c9817db442 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91644) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:25:11.823951 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device a6b8dc33-0ff3-46bf-a09b-b512222443bb up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:25:11.899589 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port a6b8dc33-0ff3-46bf-a09b-b512222443bb completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:25:11.904424 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port a6b8dc33-0ff3-46bf-a09b-b512222443bb triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:11.904803 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:11.962650 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-0a7a16d5-9b20-4b9e-aec6-e1aa51febf3d tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:25:11.975517 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e28aa821-870e-4f39-bd92-45676718bd6a None None] Sync routers for ids ['10aeccef-e762-4811-a411-a1c9817db442'] in dv-cinder-782405-9-iscsi {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:25:11.985405 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0a7a16d5-9b20-4b9e-aec6-e1aa51febf3d tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:11.990664 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e28aa821-870e-4f39-bd92-45676718bd6a None None] Routers {'10aeccef-e762-4811-a411-a1c9817db442'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:25:11.991446 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-e28aa821-870e-4f39-bd92-45676718bd6a None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:24:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['10aeccef-e762-4811-a411-a1c9817db442'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:12.020228 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:12.036757 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:12.123682 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-0a7a16d5-9b20-4b9e-aec6-e1aa51febf3d tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "PUT /v2.0/floatingips/b1bce8cf-eac1-4b98-800e-1ec838d5f270 HTTP/1.1" status: 200 len: 991 time: 3.0081110 May 20 04:25:12.126126 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b', 'tag': '2cf59c2c-723c-4450-8b98-0809b16290a8'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:25:12.158961 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port a6b8dc33-0ff3-46bf-a09b-b512222443bb, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=a6b8dc33-0ff3-46bf-a09b-b512222443bb,segment=NetworkSegment(c7d77442-5ac6-48cf-ab59-3f5962fc1e3e),segment_id=c7d77442-5ac6-48cf-ab59-3f5962fc1e3e)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:25:12.169308 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:12.175508 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-1dec07d9-a5dc-4381-b5a9-70c20f811682 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:12.254391 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-e28aa821-870e-4f39-bd92-45676718bd6a None None] Set router 10aeccef-e762-4811-a411-a1c9817db442 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:25:12.283177 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=a6b8dc33-0ff3-46bf-a09b-b512222443bb,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:12.294140 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.119s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:12.360045 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cc45fa63-bee4-47f0-9dcf-88daa88c72ce req-4dbcf0e5-d8b5-4cad-b515-1a2ab72c7f06 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:12.361669 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cc45fa63-bee4-47f0-9dcf-88daa88c72ce req-4dbcf0e5-d8b5-4cad-b515-1a2ab72c7f06 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=bec56d8703cf47f3b7193b7d1f55396d&device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 1098 time: 0.2149377 May 20 04:25:12.412974 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-94264c1b-3b3c-46dc-baff-14c5decd5523 req-ac620b74-4f11-4985-8bd7-ec177f0554f2 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:12.414581 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-94264c1b-3b3c-46dc-baff-14c5decd5523 req-ac620b74-4f11-4985-8bd7-ec177f0554f2 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1932619 May 20 04:25:12.442851 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-8071551', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-8053391'] for router, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:12.604388 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-94264c1b-3b3c-46dc-baff-14c5decd5523 req-98a8b106-6c74-4d11-9c88-0d91a63d6726 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1673048 May 20 04:25:12.607370 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-8ddc34c4-2c89-4bfa-bfde-250336cf72ff None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:12.637452 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cc45fa63-bee4-47f0-9dcf-88daa88c72ce req-601971ad-8841-4d48-ad72-38828b771e4f service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:12.639137 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cc45fa63-bee4-47f0-9dcf-88daa88c72ce req-601971ad-8841-4d48-ad72-38828b771e4f service neutron] 10.222.0.74 "GET /v2.0/networks?id=36b37213-8439-447d-9eba-a78baf7c698e HTTP/1.1" status: 200 len: 920 time: 0.2672768 May 20 04:25:12.650384 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-46b8a492-d4e4-4736-8b3e-b094ca861f5a req-0878ec37-1879-4080-8671-ca91c5704502 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:12.657615 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-46b8a492-d4e4-4736-8b3e-b094ca861f5a req-0878ec37-1879-4080-8671-ca91c5704502 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 943 time: 0.2552650 May 20 04:25:12.667413 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Router 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 is scheduled to L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 with binding_index 1 {{(pid=91644) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 20 04:25:12.714716 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-e28aa821-870e-4f39-bd92-45676718bd6a None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33'}], 'enable_snat': True}, 'gw_port_id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:24:36Z', 'revision_number': 4, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port': {'id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:64:1f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_gateway', 'standard_attr_id': 295, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:31Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'b1bce8cf-eac1-4b98-800e-1ec838d5f270', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'floating_ip_address': '172.27.0.159', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '10aeccef-e762-4811-a411-a1c9817db442', 'port_id': '2cf59c2c-723c-4450-8b98-0809b16290a8', 'fixed_ip_address': '172.20.0.3', 'status': 'DOWN', 'standard_attr_id': 320, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'mac_address': 'fa:16:3e:b8:b0:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:24:40Z', 'updated_at': '2021-05-20T04:25:10Z', 'revision_number': 1, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '7dc69379-dd61-4add-8e0a-4ef00a211a0a', 'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'mac_address': 'fa:16:3e:a4:49:d4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_interface', 'standard_attr_id': 307, 'fixed_ips': [{'subnet_id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:35Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 5, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'subnets': [{'id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:12.722141 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e28aa821-870e-4f39-bd92-45676718bd6a None None] Checking router: 10aeccef-e762-4811-a411-a1c9817db442 for host: dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:25:12.724295 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device a859e036-1f75-40cc-b35b-68bdda10b71f details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:25:12.732814 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e28aa821-870e-4f39-bd92-45676718bd6a None None] Routers {'10aeccef-e762-4811-a411-a1c9817db442'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:25:12.733875 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-e28aa821-870e-4f39-bd92-45676718bd6a None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:24:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['10aeccef-e762-4811-a411-a1c9817db442'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:12.844071 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cc45fa63-bee4-47f0-9dcf-88daa88c72ce req-572d0627-82ec-4eb7-86bb-38d825585dca service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=706277c6-cd8c-4146-ab91-ed9504446690 HTTP/1.1" status: 200 len: 217 time: 0.0768783 May 20 04:25:12.915780 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-46b8a492-d4e4-4736-8b3e-b094ca861f5a req-5aaae295-00bb-40f0-85ed-98b965edd321 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2569325 May 20 04:25:12.915930 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b', 'tag': '2cf59c2c-723c-4450-8b98-0809b16290a8', 'status': 'completed', 'code': 200} May 20 04:25:12.916845 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:25:12.934934 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port a859e036-1f75-40cc-b35b-68bdda10b71f, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=a859e036-1f75-40cc-b35b-68bdda10b71f,segment=NetworkSegment(6fe99325-a72c-48bf-abc7-9bbcc4b753ff),segment_id=6fe99325-a72c-48bf-abc7-9bbcc4b753ff)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:25:12.945274 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Sync routers for ids ['5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:25:12.947962 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-e28aa821-870e-4f39-bd92-45676718bd6a None None] Set router 10aeccef-e762-4811-a411-a1c9817db442 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:25:12.963306 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cc45fa63-bee4-47f0-9dcf-88daa88c72ce req-818a556e-58d3-4cd7-9940-06d79d4497b5 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:12.965324 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cc45fa63-bee4-47f0-9dcf-88daa88c72ce req-818a556e-58d3-4cd7-9940-06d79d4497b5 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=b25ae9c8-e11d-45ee-bde2-332cc1538602 HTTP/1.1" status: 200 len: 859 time: 0.1148489 May 20 04:25:12.965445 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Routers {'5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:25:12.968069 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:24:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:12.997131 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Notify callbacks [] for router, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:13.001304 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] 10.222.0.39 "POST /v2.0/routers HTTP/1.1" status: 201 len: 920 time: 4.2463479 May 20 04:25:13.003281 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 37664) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:13.025944 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cc45fa63-bee4-47f0-9dcf-88daa88c72ce req-7102c294-ce7a-4466-85ea-90b68a314273 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=36b37213-8439-447d-9eba-a78baf7c698e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0538056 May 20 04:25:13.032885 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': 'a859e036-1f75-40cc-b35b-68bdda10b71f', 'network_id': '54fc8d38-193b-49c4-b873-c5353723791f', 'port_id': 'a859e036-1f75-40cc-b35b-68bdda10b71f', 'mac_address': 'fa:16:3e:f8:c9:fa', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 836, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '0f918085-8d33-4e36-a59e-7f270a2b1874', 'ip_address': '172.20.0.4'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:25:13.122439 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:13.144235 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id a859e036-1f75-40cc-b35b-68bdda10b71f {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:25:13.160006 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port a859e036-1f75-40cc-b35b-68bdda10b71f, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=a859e036-1f75-40cc-b35b-68bdda10b71f,segment=NetworkSegment(6fe99325-a72c-48bf-abc7-9bbcc4b753ff),segment_id=6fe99325-a72c-48bf-abc7-9bbcc4b753ff)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:25:13.169817 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:13.172708 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4603bb9f-b74e-4916-9c6d-82d70e75fe8e req-52b2b39e-718c-4078-be16-cd2ad4d9ffac service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:13.173833 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-8eed0b07-ca9a-4558-ad1c-f21ff862419c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:13.174208 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-4603bb9f-b74e-4916-9c6d-82d70e75fe8e req-52b2b39e-718c-4078-be16-cd2ad4d9ffac service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=198bc8ab083f4a729f32b8ef27a6ee77&device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 1098 time: 0.2627056 May 20 04:25:13.193469 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cc45fa63-bee4-47f0-9dcf-88daa88c72ce req-fe8d2b6b-f4e4-414b-b8c7-84df2f731f44 service neutron] 10.222.0.74 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1593037 May 20 04:25:13.325746 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=a859e036-1f75-40cc-b35b-68bdda10b71f,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:13.331065 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.157s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:13.332878 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Set router 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:25:13.364822 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-e28aa821-870e-4f39-bd92-45676718bd6a None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33'}], 'enable_snat': True}, 'gw_port_id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:24:36Z', 'revision_number': 4, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port': {'id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:64:1f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_gateway', 'standard_attr_id': 295, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:31Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'b1bce8cf-eac1-4b98-800e-1ec838d5f270', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'floating_ip_address': '172.27.0.159', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '10aeccef-e762-4811-a411-a1c9817db442', 'port_id': '2cf59c2c-723c-4450-8b98-0809b16290a8', 'fixed_ip_address': '172.20.0.3', 'status': 'DOWN', 'standard_attr_id': 320, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'mac_address': 'fa:16:3e:b8:b0:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:24:40Z', 'updated_at': '2021-05-20T04:25:10Z', 'revision_number': 1, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '7dc69379-dd61-4add-8e0a-4ef00a211a0a', 'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'mac_address': 'fa:16:3e:a4:49:d4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_interface', 'standard_attr_id': 307, 'fixed_ips': [{'subnet_id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:35Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 5, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'subnets': [{'id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:13.369754 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e28aa821-870e-4f39-bd92-45676718bd6a None None] The sync data for ids ['10aeccef-e762-4811-a411-a1c9817db442'] in dv-cinder-782405-9-iscsi is: [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33'}], 'enable_snat': True}, 'gw_port_id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:24:36Z', 'revision_number': 4, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port': {'id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:64:1f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_gateway', 'standard_attr_id': 295, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:31Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'b1bce8cf-eac1-4b98-800e-1ec838d5f270', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'floating_ip_address': '172.27.0.159', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '10aeccef-e762-4811-a411-a1c9817db442', 'port_id': '2cf59c2c-723c-4450-8b98-0809b16290a8', 'fixed_ip_address': '172.20.0.3', 'status': 'DOWN', 'standard_attr_id': 320, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'mac_address': 'fa:16:3e:b8:b0:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:24:40Z', 'updated_at': '2021-05-20T04:25:10Z', 'revision_number': 1, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '7dc69379-dd61-4add-8e0a-4ef00a211a0a', 'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'mac_address': 'fa:16:3e:a4:49:d4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_interface', 'standard_attr_id': 307, 'fixed_ips': [{'subnet_id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:35Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 5, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'subnets': [{'id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:25:13.379392 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cc45fa63-bee4-47f0-9dcf-88daa88c72ce req-9e19d703-5c12-4ea1-b0e7-5c08441ae62d service neutron] 10.222.0.74 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1733024 May 20 04:25:13.633761 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-58e5b57a-4e2a-48db-89a7-9fb2f090d319 None None] New status for floating IP b1bce8cf-eac1-4b98-800e-1ec838d5f270: ACTIVE {{(pid=91646) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 20 04:25:13.650277 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'name': 'tempest-TestShelveInstance-251001111-router', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100'}], 'enable_snat': True}, 'gw_port_id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:08Z', 'updated_at': '2021-05-20T04:25:11Z', 'revision_number': 3, 'project_id': '39febf983bf24778a300f287dbeca829', 'gw_port': {'id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:d3:05:8a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:10Z', 'updated_at': '2021-05-20T04:25:10Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:13.654323 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Checking router: 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:25:13.654680 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Updating router 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 port 37393b04-8a64-4737-953b-1c02d4efce9f binding host dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 20 04:25:13.688244 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4603bb9f-b74e-4916-9c6d-82d70e75fe8e req-4b9ddddc-cd8e-44fa-8d47-5ef82b71509e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:13.689679 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-4603bb9f-b74e-4916-9c6d-82d70e75fe8e req-4b9ddddc-cd8e-44fa-8d47-5ef82b71509e service neutron] 10.222.0.74 "GET /v2.0/networks?id=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d HTTP/1.1" status: 200 len: 910 time: 0.5057981 May 20 04:25:13.723760 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:13.869543 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-4603bb9f-b74e-4916-9c6d-82d70e75fe8e req-777bc3cd-1803-423e-b470-02c76f03bef1 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=2cf59c2c-723c-4450-8b98-0809b16290a8 HTTP/1.1" status: 200 len: 999 time: 0.1674993 May 20 04:25:13.889733 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b3727c95-e989-4334-adbf-e082aba60a2d req-f00174bb-35c9-4f50-8605-3b3bfdbd5c42 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:13.904128 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b3727c95-e989-4334-adbf-e082aba60a2d req-f00174bb-35c9-4f50-8605-3b3bfdbd5c42 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1745410 May 20 04:25:13.981697 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b3727c95-e989-4334-adbf-e082aba60a2d req-455a29f1-3171-4cd3-b4ac-f8d215ffe49d tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0762455 May 20 04:25:14.013253 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4603bb9f-b74e-4916-9c6d-82d70e75fe8e req-ed09d5a9-fd70-4aba-b67f-59adaf6ccba5 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:14.014715 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-4603bb9f-b74e-4916-9c6d-82d70e75fe8e req-ed09d5a9-fd70-4aba-b67f-59adaf6ccba5 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3 HTTP/1.1" status: 200 len: 848 time: 0.1366367 May 20 04:25:14.057385 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['a859e036-1f75-40cc-b35b-68bdda10b71f'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:25:14.083862 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device a6b8dc33-0ff3-46bf-a09b-b512222443bb up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:25:14.154646 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d4f4c73a-24ad-4185-852e-63d696c6d497 req-9ff421ee-f018-4565-954b-328e27ccccc8 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:14.156055 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d4f4c73a-24ad-4185-852e-63d696c6d497 req-9ff421ee-f018-4565-954b-328e27ccccc8 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 944 time: 0.0892491 May 20 04:25:14.189516 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Notify callbacks [] for port, precommit_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:14.198553 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4603bb9f-b74e-4916-9c6d-82d70e75fe8e req-592651cf-4504-47c2-bce6-46d62eec3a2c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:14.200038 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-4603bb9f-b74e-4916-9c6d-82d70e75fe8e req-592651cf-4504-47c2-bce6-46d62eec3a2c service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1748006 May 20 04:25:14.214483 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Transition to ACTIVE for port object 37393b04-8a64-4737-953b-1c02d4efce9f will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:25:14.216097 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d4f4c73a-24ad-4185-852e-63d696c6d497 req-90e7845d-c550-480b-a56f-834270301681 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0530431 May 20 04:25:14.230544 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:14.237230 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:14.366802 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port a6b8dc33-0ff3-46bf-a09b-b512222443bb triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:14.367173 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:14.370637 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:14.401211 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Attempting to bind port 37393b04-8a64-4737-953b-1c02d4efce9f on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:25:14.401585 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Attempting to bind port 37393b04-8a64-4737-953b-1c02d4efce9f by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:25:14.401949 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Attempting to bind port 37393b04-8a64-4737-953b-1c02d4efce9f on network 712ce8c4-8feb-4719-bf13-e06992125cd9 {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:25:14.402896 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-4603bb9f-b74e-4916-9c6d-82d70e75fe8e req-8681f32b-a71a-4b3a-9993-570dfb528330 service neutron] 10.222.0.74 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1909754 May 20 04:25:14.411361 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e730247e-f308-4816-8b4d-1425b012cfe5 None None] Pushing event updated for resources: {'Port': ['ID=37393b04-8a64-4737-953b-1c02d4efce9f,revision_number=2']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:14.414244 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-e730247e-f308-4816-8b4d-1425b012cfe5 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.177s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:14.438416 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 24, 45), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:25:14.438865 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Checking segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:25:14.439205 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:14.439597 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Bound using segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:25:14.440106 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Bound port: 37393b04-8a64-4737-953b-1c02d4efce9f, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:25:14.440502 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:14.622501 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] For port 37393b04-8a64-4737-953b-1c02d4efce9f, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:25:14.641932 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-4603bb9f-b74e-4916-9c6d-82d70e75fe8e req-e87e7dac-d43c-4392-886b-c56b3ad01f15 service neutron] 10.222.0.74 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2019937 May 20 04:25:14.644814 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] For port 37393b04-8a64-4737-953b-1c02d4efce9f, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=37393b04-8a64-4737-953b-1c02d4efce9f,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:25:14.650946 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:14.654957 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:14.698035 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Allocated IP 172.20.0.1 (e57f6e29-4637-455e-af9d-0975c45c995c/49523a89-260b-4732-abff-507d74fed4c4/c7df749b-cc66-4529-a666-5c9e0a997ff8) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:25:14.733726 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Routers {'5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:25:14.734345 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:24:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:14.796261 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e730247e-f308-4816-8b4d-1425b012cfe5 None None] Pushing event updated for resources: {'Port': ['ID=37393b04-8a64-4737-953b-1c02d4efce9f,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:14.799463 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-e730247e-f308-4816-8b4d-1425b012cfe5 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.144s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:14.844822 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:14.861611 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Transition to ACTIVE for port object c7df749b-cc66-4529-a666-5c9e0a997ff8 will not be triggered until provisioned by entity DHCP. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:25:14.885530 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:14.890216 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e0314cc0-78bb-4574-bf6a-9f762ae16e1c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:14.915381 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] Set router 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:25:14.953410 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Network e57f6e29-4637-455e-af9d-0975c45c995c is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:25:14.956240 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:25:14.959867 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:25:14.996487 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Pushing event updated for resources: {'Port': ['ID=c7df749b-cc66-4529-a666-5c9e0a997ff8,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:15.005177 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.115s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:15.020811 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-8003132'] for router_interface, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:15.112796 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:15.208227 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-5dc3d17e-ab41-40c6-88d8-b2105c8be063 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:15.225686 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fadce3d9-425e-4007-b950-ef3d97d7ff16 req-8204e02d-e849-4c82-adfc-59cf4c101ba4 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:15.227128 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fadce3d9-425e-4007-b950-ef3d97d7ff16 req-8204e02d-e849-4c82-adfc-59cf4c101ba4 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1159904 May 20 04:25:15.281519 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:15.288244 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fadce3d9-425e-4007-b950-ef3d97d7ff16 req-ff067c01-40b2-4235-8af0-aa96774ded2d tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0551753 May 20 04:25:15.310737 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-b7eae0e0-0b08-4f7a-bdd2-601d65d53123 None None] Provisioning for port c7df749b-cc66-4529-a666-5c9e0a997ff8 completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:25:15.316798 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-b7eae0e0-0b08-4f7a-bdd2-601d65d53123 None None] Provisioning complete for port c7df749b-cc66-4529-a666-5c9e0a997ff8 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:15.317174 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-b7eae0e0-0b08-4f7a-bdd2-601d65d53123 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:15.374824 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-b7eae0e0-0b08-4f7a-bdd2-601d65d53123 None None] Port c7df749b-cc66-4529-a666-5c9e0a997ff8 cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:25:15.380539 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'name': 'tempest-TestShelveInstance-251001111-router', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100'}], 'enable_snat': True}, 'gw_port_id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:08Z', 'updated_at': '2021-05-20T04:25:11Z', 'revision_number': 3, 'project_id': '39febf983bf24778a300f287dbeca829', 'gw_port': {'id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:d3:05:8a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:10Z', 'updated_at': '2021-05-20T04:25:14Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:15.384634 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3668aef0-2a13-4dc3-94cd-e335099a9919 None None] The sync data for ids ['5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'] in dv-cinder-782405-9-iscsi is: [{'id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'name': 'tempest-TestShelveInstance-251001111-router', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100'}], 'enable_snat': True}, 'gw_port_id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:08Z', 'updated_at': '2021-05-20T04:25:11Z', 'revision_number': 3, 'project_id': '39febf983bf24778a300f287dbeca829', 'gw_port': {'id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:d3:05:8a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:10Z', 'updated_at': '2021-05-20T04:25:14Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:25:15.417753 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-40daeb17-39e7-486f-9da3-001e4a125d04 req-a34e8fc6-ee77-40ca-8379-f22cba95e492 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:15.419430 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-40daeb17-39e7-486f-9da3-001e4a125d04 req-a34e8fc6-ee77-40ca-8379-f22cba95e492 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 944 time: 0.0893435 May 20 04:25:15.466270 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-40daeb17-39e7-486f-9da3-001e4a125d04 req-bda356c7-5590-45da-901e-50bd3915692b tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0401468 May 20 04:25:15.517141 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:15.524047 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:15.547802 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:15.639329 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Pushing event updated for resources: {'Port': ['ID=c7df749b-cc66-4529-a666-5c9e0a997ff8,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:15.642481 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-7924897', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-8003163'] for router_interface, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:15.648832 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.101s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:15.816427 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Network e57f6e29-4637-455e-af9d-0975c45c995c is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:25:15.817579 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:25:15.844728 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 37393b04-8a64-4737-953b-1c02d4efce9f up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:25:15.933122 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port 37393b04-8a64-4737-953b-1c02d4efce9f completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:25:15.939036 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 37393b04-8a64-4737-953b-1c02d4efce9f triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:15.939467 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:16.031527 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Router 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91644) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:25:16.093810 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:16.116442 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port 37393b04-8a64-4737-953b-1c02d4efce9f, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=37393b04-8a64-4737-953b-1c02d4efce9f,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:25:16.117552 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-5c24e117-6bb6-4189-9e71-5188ae6388a5 None None] Provisioning complete for port c7df749b-cc66-4529-a666-5c9e0a997ff8 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:16.118403 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-5c24e117-6bb6-4189-9e71-5188ae6388a5 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:16.147914 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:16.158463 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-aa5aee1a-df53-4787-80b5-0ed18ba9a138 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:16.219423 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:25:16.229242 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] 10.222.0.39 "PUT /v2.0/routers/5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66/add_router_interface HTTP/1.1" status: 200 len: 503 time: 3.2249744 May 20 04:25:16.250611 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-5c24e117-6bb6-4189-9e71-5188ae6388a5 None None] Port c7df749b-cc66-4529-a666-5c9e0a997ff8 cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:25:16.257416 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:16.260990 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=37393b04-8a64-4737-953b-1c02d4efce9f,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:16.264315 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.106s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:16.467251 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b7f3a80e-2c87-43fc-88f0-a483cb27620b req-db971581-e616-4f26-a962-1062904963a6 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:16.468908 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b7f3a80e-2c87-43fc-88f0-a483cb27620b req-db971581-e616-4f26-a962-1062904963a6 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0808568 May 20 04:25:16.519443 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b7f3a80e-2c87-43fc-88f0-a483cb27620b req-5126eef3-74c7-4a2d-8e13-fa09299bd7c6 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0444825 May 20 04:25:16.662841 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29fdc1aa-27e4-40fb-922f-6441f752394a req-e42e8b8c-ffd6-49e2-bcd6-be35ec29ce53 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:16.664424 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-29fdc1aa-27e4-40fb-922f-6441f752394a req-e42e8b8c-ffd6-49e2-bcd6-be35ec29ce53 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 944 time: 0.0802615 May 20 04:25:16.723751 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-29fdc1aa-27e4-40fb-922f-6441f752394a req-27d95295-dd82-491b-bb03-efb8582d2591 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0478013 May 20 04:25:16.725755 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device a859e036-1f75-40cc-b35b-68bdda10b71f up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:25:16.766623 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 37716) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:16.773069 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Request body: {'network': {'name': 'tempest-TestShelveInstance-2138875911-network', 'tenant_id': '03fbb7d8cfaa4c2ab40ef558c109e377'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:16.777690 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:16.781614 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Usage tracker for resource:network and tenant:03fbb7d8cfaa4c2ab40ef558c109e377 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:25:16.784642 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:25:16.790397 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:16.800914 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Made reservation on behalf of 03fbb7d8cfaa4c2ab40ef558c109e377 for: {'network': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:25:16.803415 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-585852'] for network, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:16.813138 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port a859e036-1f75-40cc-b35b-68bdda10b71f completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:25:16.815764 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:16.818252 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port a859e036-1f75-40cc-b35b-68bdda10b71f triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:16.818515 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:16.832117 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:16.836299 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Usage tracker for resource:security_group_rule and tenant:03fbb7d8cfaa4c2ab40ef558c109e377 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:25:16.839073 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:25:16.845140 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:16.958489 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:16.959129 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:16.963804 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:16.967363 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-465f8d8e-ef89-4d7a-b889-543212f92987 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:16.986097 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '2f093c2c-1922-4116-95f2-a2e99555016d', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'a859e036-1f75-40cc-b35b-68bdda10b71f'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:25:16.995754 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port a859e036-1f75-40cc-b35b-68bdda10b71f, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=a859e036-1f75-40cc-b35b-68bdda10b71f,segment=NetworkSegment(6fe99325-a72c-48bf-abc7-9bbcc4b753ff),segment_id=6fe99325-a72c-48bf-abc7-9bbcc4b753ff)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:25:16.997370 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e1a100be-6b01-4c74-80a6-a3bd788d8ca7,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:17.000921 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.034s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:17.002609 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:17.004720 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:17.005493 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 840} {{(pid=91643) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 20 04:25:17.016632 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:17.017054 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.segments_db [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Added segment 6822d26b-245e-4501-a6f1-7f56846f560c of type vlan for network c83ca109-2f8b-40c8-b2dd-b471ffdb582f May 20 04:25:17.051894 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8760688516720'] for network, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:17.054903 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '2f093c2c-1922-4116-95f2-a2e99555016d', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'a859e036-1f75-40cc-b35b-68bdda10b71f', 'code': 200} May 20 04:25:17.074674 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=a859e036-1f75-40cc-b35b-68bdda10b71f,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:17.077562 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.073s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:17.091034 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915'] for network, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:17.093237 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:17.093868 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:17.099271 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-02bb02bd-a7d5-4126-b510-e5be30fc9a22 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:17.133954 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Sync routers for ids ['5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:25:17.144568 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Routers {'5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:25:17.145413 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 6, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 6, "log_agent_heartbeats": false, "routers": 6},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:25:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:17.203911 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Pushing event updated for resources: {'Network': ['ID=c83ca109-2f8b-40c8-b2dd-b471ffdb582f,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:17.208124 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.109s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:17.213184 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 37726) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:17.215926 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] 10.222.0.39 "POST /v2.0/networks HTTP/1.1" status: 201 len: 882 time: 0.4480748 May 20 04:25:17.219416 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-ee838674-9e9f-4e72-b5b4-fd48df802b5b admin admin] Request body: {'subnet': {'network_id': 'c83ca109-2f8b-40c8-b2dd-b471ffdb582f', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestShelveInstance-2138875911-subnet', 'tenant_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'ip_version': 4}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:17.224451 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-ee838674-9e9f-4e72-b5b4-fd48df802b5b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:17.229437 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-ee838674-9e9f-4e72-b5b4-fd48df802b5b admin admin] Usage tracker for resource:subnet and tenant:03fbb7d8cfaa4c2ab40ef558c109e377 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:25:17.232414 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-ee838674-9e9f-4e72-b5b4-fd48df802b5b admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:25:17.235836 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Network c83ca109-2f8b-40c8-b2dd-b471ffdb582f details requested by agent ovs-agent-dv-cinder-782405-9-iscsi with host dv-cinder-782405-9-iscsi {{(pid=91645) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 20 04:25:17.239397 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-ee838674-9e9f-4e72-b5b4-fd48df802b5b admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:17.279362 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-ee838674-9e9f-4e72-b5b4-fd48df802b5b admin admin] Made reservation on behalf of 03fbb7d8cfaa4c2ab40ef558c109e377 for: {'subnet': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:25:17.281494 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ee838674-9e9f-4e72-b5b4-fd48df802b5b admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-1810190'] for subnet, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:17.412156 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-200e77a0-0099-4496-8ff0-ad24191c25b6 req-15baaa38-da7b-4664-b6d3-0832f6a51a1e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:17.413632 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-200e77a0-0099-4496-8ff0-ad24191c25b6 req-15baaa38-da7b-4664-b6d3-0832f6a51a1e service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 1098 time: 0.1680486 May 20 04:25:17.448707 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Set router 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:25:17.503017 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-200e77a0-0099-4496-8ff0-ad24191c25b6 req-abddae31-96af-4091-a03e-587f6e9e96e8 service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0814004 May 20 04:25:17.714698 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-99583aa5-a962-410f-851f-f4702a47eb0a req-b9d37357-a2f2-4fe8-87b4-98e8caf748bb tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:17.716192 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ee838674-9e9f-4e72-b5b4-fd48df802b5b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-1988978', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-1810236'] for subnet, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:17.716333 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-99583aa5-a962-410f-851f-f4702a47eb0a req-b9d37357-a2f2-4fe8-87b4-98e8caf748bb tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0892341 May 20 04:25:17.720356 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:17.785562 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-99583aa5-a962-410f-851f-f4702a47eb0a req-d07c3f7a-e479-4612-955e-e1f94a213226 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0555949 May 20 04:25:17.809091 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ee838674-9e9f-4e72-b5b4-fd48df802b5b admin admin] Pushing event updated for resources: {'Subnet': ['ID=747296cc-d241-444d-9f89-67006bc5546d,revision_number=0']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:17.813059 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-ee838674-9e9f-4e72-b5b4-fd48df802b5b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.093s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:17.820914 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ee838674-9e9f-4e72-b5b4-fd48df802b5b admin admin] Checking segment: {'id': '6822d26b-245e-4501-a6f1-7f56846f560c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 840, 'network_id': 'c83ca109-2f8b-40c8-b2dd-b471ffdb582f'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:25:17.820914 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-ee838674-9e9f-4e72-b5b4-fd48df802b5b admin admin] Network hostable DHCP agents. Network: c83ca109-2f8b-40c8-b2dd-b471ffdb582f, hostable agents: ['fffae25a-f450-4182-8f9e-ac29d143f6c9'], hosted agents: [] {{(pid=91644) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 20 04:25:17.822776 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'name': 'tempest-TestShelveInstance-251001111-router', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100'}], 'enable_snat': True}, 'gw_port_id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:08Z', 'updated_at': '2021-05-20T04:25:15Z', 'revision_number': 4, 'project_id': '39febf983bf24778a300f287dbeca829', 'gw_port': {'id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:d3:05:8a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:10Z', 'updated_at': '2021-05-20T04:25:16Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'c7df749b-cc66-4529-a666-5c9e0a997ff8', 'name': '', 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'mac_address': 'fa:16:3e:8f:27:b0', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_interface', 'standard_attr_id': 355, 'fixed_ips': [{'subnet_id': '49523a89-260b-4732-abff-507d74fed4c4', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:14Z', 'updated_at': '2021-05-20T04:25:15Z', 'revision_number': 2, 'project_id': '39febf983bf24778a300f287dbeca829', 'subnets': [{'id': '49523a89-260b-4732-abff-507d74fed4c4', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:17.826440 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:17.827909 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Checking router: 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:25:17.828266 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Updating router 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 port c7df749b-cc66-4529-a666-5c9e0a997ff8 binding host dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 20 04:25:17.840876 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-ee838674-9e9f-4e72-b5b4-fd48df802b5b admin admin] Network c83ca109-2f8b-40c8-b2dd-b471ffdb582f is scheduled to be hosted by DHCP agent fffae25a-f450-4182-8f9e-ac29d143f6c9 with binding_index 1 {{(pid=91644) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 20 04:25:17.852237 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-ee838674-9e9f-4e72-b5b4-fd48df802b5b admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-782405-9-iscsi, c83ca109-2f8b-40c8-b2dd-b471ffdb582f) {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:25:17.853184 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 37393b04-8a64-4737-953b-1c02d4efce9f up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:25:17.861835 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-ee838674-9e9f-4e72-b5b4-fd48df802b5b admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:17.862516 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ee838674-9e9f-4e72-b5b4-fd48df802b5b admin admin] Notify callbacks [] for subnet, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:17.893952 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-1d4b75c7-e3fd-495c-8d7e-7a6168669fd8 None None] Network c83ca109-2f8b-40c8-b2dd-b471ffdb582f requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:25:17.957894 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f5cd8dd8-c35d-4235-b6c6-17f69f756ba2 req-a6673e31-d0d6-4b33-a01a-d3ffc3e9375e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:17.959440 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f5cd8dd8-c35d-4235-b6c6-17f69f756ba2 req-a6673e31-d0d6-4b33-a01a-d3ffc3e9375e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 945 time: 0.0942135 May 20 04:25:17.994772 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-ee838674-9e9f-4e72-b5b4-fd48df802b5b admin admin] 10.222.0.39 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 845 time: 0.7803271 May 20 04:25:17.997133 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 37738) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:18.003171 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Request body: {'router': {'name': 'tempest-TestShelveInstance-2138875911-router', 'tenant_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:18.009019 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:18.012151 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f5cd8dd8-c35d-4235-b6c6-17f69f756ba2 req-77ded3e7-377c-4337-8d0b-13b3ec43cfd6 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0468836 May 20 04:25:18.017855 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Usage tracker for resource:router and tenant:03fbb7d8cfaa4c2ab40ef558c109e377 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:25:18.021168 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:25:18.030342 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:18.040712 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:18.051878 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-8071540', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-260712'] for router, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:18.053244 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-8056436', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-8053301', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-7881139', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-260707'] for router, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:18.070267 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:18.120959 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 37393b04-8a64-4737-953b-1c02d4efce9f triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:18.121370 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:18.332344 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-8053308', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-8071430', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-260685'] for router, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:18.440786 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcp85c0b5c8-c799-5455-967d-3e9c2c8b4839-c83ca109-2f8b-40c8-b2dd-b471ffdb582f', 'network_id': 'c83ca109-2f8b-40c8-b2dd-b471ffdb582f', 'tenant_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'fixed_ips': [{'subnet_id': '747296cc-d241-444d-9f89-67006bc5546d'}]}} from dv-cinder-782405-9-iscsi. {{(pid=91645) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 20 04:25:18.441614 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:18.637929 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:18.670518 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:18.742404 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Notify callbacks [] for port, precommit_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:18.744947 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.db_base_plugin_common [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Allocated IP 172.20.0.2 (c83ca109-2f8b-40c8-b2dd-b471ffdb582f/747296cc-d241-444d-9f89-67006bc5546d/424ccab3-7db4-445a-bbbc-4f034b243ce3) {{(pid=91645) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:25:18.761128 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Transition to ACTIVE for port object c7df749b-cc66-4529-a666-5c9e0a997ff8 will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:25:18.780581 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:18.796590 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:18.808358 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Allocated IP 172.27.0.116 (712ce8c4-8feb-4719-bf13-e06992125cd9/e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5/b57cd7ef-aeba-4bab-8870-543c319b914b) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:25:18.979469 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:18.981579 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3bb29233-0a22-482f-8eff-25f5dbf55962 req-99d8bbd1-aaf5-462b-adf6-85d23f4f43f9 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:18.984528 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3bb29233-0a22-482f-8eff-25f5dbf55962 req-99d8bbd1-aaf5-462b-adf6-85d23f4f43f9 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0913963 May 20 04:25:19.012576 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a14e1d56-4a2f-40ff-aa8a-2790ce9d2ba7 None None] Pushing event updated for resources: {'Port': ['ID=c7df749b-cc66-4529-a666-5c9e0a997ff8,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:19.022099 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-a14e1d56-4a2f-40ff-aa8a-2790ce9d2ba7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.225s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:19.025287 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:19.036841 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-1c784839-874f-48d4-9897-af7f8122b6ae tempest-DeleteServersTestJSON-745374672 tempest-DeleteServersTestJSON-745374672-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:19.092512 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:25:19.101476 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:25:19.104135 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:25:19.137573 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Attempting to bind port c7df749b-cc66-4529-a666-5c9e0a997ff8 on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:25:19.137866 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Attempting to bind port c7df749b-cc66-4529-a666-5c9e0a997ff8 by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': 'c7d77442-5ac6-48cf-ab59-3f5962fc1e3e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 806, 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:25:19.138233 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Attempting to bind port c7df749b-cc66-4529-a666-5c9e0a997ff8 on network e57f6e29-4637-455e-af9d-0975c45c995c {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:25:19.162232 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Pushing event updated for resources: {'Port': ['ID=b57cd7ef-aeba-4bab-8870-543c319b914b,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:19.170842 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.134s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:19.192106 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 25, 15), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:25:19.192390 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Checking segment: {'id': 'c7d77442-5ac6-48cf-ab59-3f5962fc1e3e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 806, 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:25:19.192975 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:19.193521 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Bound using segment: {'id': 'c7d77442-5ac6-48cf-ab59-3f5962fc1e3e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 806, 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:25:19.194164 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Bound port: c7df749b-cc66-4529-a666-5c9e0a997ff8, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'c7d77442-5ac6-48cf-ab59-3f5962fc1e3e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 806, 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:25:19.194589 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3bb29233-0a22-482f-8eff-25f5dbf55962 req-2f41b63e-5293-4d48-a771-3a58acc459c8 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2036519 May 20 04:25:19.195107 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:19.288840 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Notify callbacks [] for port, precommit_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:19.313708 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Transition to ACTIVE for port object 424ccab3-7db4-445a-bbbc-4f034b243ce3 will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:25:19.319331 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f55dfaa7-9bd2-4334-8c8a-6fd9148a0472 req-e0bb6933-194e-4e22-aa35-296d56751326 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:19.322646 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f55dfaa7-9bd2-4334-8c8a-6fd9148a0472 req-e0bb6933-194e-4e22-aa35-296d56751326 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 945 time: 0.1011963 May 20 04:25:19.328353 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] For port c7df749b-cc66-4529-a666-5c9e0a997ff8, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:25:19.333796 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] For port c7df749b-cc66-4529-a666-5c9e0a997ff8, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=c7df749b-cc66-4529-a666-5c9e0a997ff8,segment=NetworkSegment(c7d77442-5ac6-48cf-ab59-3f5962fc1e3e),segment_id=c7d77442-5ac6-48cf-ab59-3f5962fc1e3e)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:25:19.340233 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Transition to ACTIVE for port object 424ccab3-7db4-445a-bbbc-4f034b243ce3 will not be triggered until provisioned by entity DHCP. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:25:19.343897 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:19.348458 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:19.379506 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f55dfaa7-9bd2-4334-8c8a-6fd9148a0472 req-053abf85-e848-45ee-bac4-e66d85109f3e tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0506802 May 20 04:25:19.395059 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:19.440905 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Network c83ca109-2f8b-40c8-b2dd-b471ffdb582f is already hosted by enough agents. {{(pid=91645) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:25:19.445942 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.base_scheduler [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Resources bound (agent type, host, resource id): {{(pid=91645) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:25:19.452133 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Received port after_create {{(pid=91645) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:25:19.454263 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a14e1d56-4a2f-40ff-aa8a-2790ce9d2ba7 None None] Pushing event updated for resources: {'Port': ['ID=c7df749b-cc66-4529-a666-5c9e0a997ff8,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:19.458284 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Routers {'5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:25:19.458904 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 6, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 6, "log_agent_heartbeats": false, "routers": 6},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:25:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:19.533749 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-a14e1d56-4a2f-40ff-aa8a-2790ce9d2ba7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.185s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:19.536073 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.135s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:19.675816 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:19.701062 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Pushing event updated for resources: {'Port': ['ID=424ccab3-7db4-445a-bbbc-4f034b243ce3,revision_number=1']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:19.709504 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.173s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:19.771198 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Attempting to bind port 424ccab3-7db4-445a-bbbc-4f034b243ce3 on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:25:19.771597 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Attempting to bind port 424ccab3-7db4-445a-bbbc-4f034b243ce3 by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '6822d26b-245e-4501-a6f1-7f56846f560c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 840, 'network_id': 'c83ca109-2f8b-40c8-b2dd-b471ffdb582f'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:25:19.771966 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Attempting to bind port 424ccab3-7db4-445a-bbbc-4f034b243ce3 on network c83ca109-2f8b-40c8-b2dd-b471ffdb582f {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:25:19.788512 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 25, 15), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:25:19.788888 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Checking segment: {'id': '6822d26b-245e-4501-a6f1-7f56846f560c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 840, 'network_id': 'c83ca109-2f8b-40c8-b2dd-b471ffdb582f'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:25:19.789300 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:19.789695 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Bound using segment: {'id': '6822d26b-245e-4501-a6f1-7f56846f560c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 840, 'network_id': 'c83ca109-2f8b-40c8-b2dd-b471ffdb582f'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:25:19.790168 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Bound port: 424ccab3-7db4-445a-bbbc-4f034b243ce3, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '6822d26b-245e-4501-a6f1-7f56846f560c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 840, 'network_id': 'c83ca109-2f8b-40c8-b2dd-b471ffdb582f'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:25:19.790546 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:19.869533 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] Set router 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:25:20.021103 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-8071551', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-8053391'] for router, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:20.036833 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] For port 424ccab3-7db4-445a-bbbc-4f034b243ce3, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:25:20.058886 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] For port 424ccab3-7db4-445a-bbbc-4f034b243ce3, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=424ccab3-7db4-445a-bbbc-4f034b243ce3,segment=NetworkSegment(6822d26b-245e-4501-a6f1-7f56846f560c),segment_id=6822d26b-245e-4501-a6f1-7f56846f560c)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:25:20.102442 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:20.110098 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:20.234743 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Router ecb83b2f-735e-4418-8208-a3dfb9673052 is scheduled to L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 with binding_index 1 {{(pid=91644) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 20 04:25:20.280740 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Pushing event updated for resources: {'Port': ['ID=424ccab3-7db4-445a-bbbc-4f034b243ce3,revision_number=2']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:20.286598 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.176s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:20.344169 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'name': 'tempest-TestShelveInstance-251001111-router', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100'}], 'enable_snat': True}, 'gw_port_id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:08Z', 'updated_at': '2021-05-20T04:25:15Z', 'revision_number': 4, 'project_id': '39febf983bf24778a300f287dbeca829', 'gw_port': {'id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:d3:05:8a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:10Z', 'updated_at': '2021-05-20T04:25:16Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'c7df749b-cc66-4529-a666-5c9e0a997ff8', 'name': '', 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'mac_address': 'fa:16:3e:8f:27:b0', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_interface', 'standard_attr_id': 355, 'fixed_ips': [{'subnet_id': '49523a89-260b-4732-abff-507d74fed4c4', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:14Z', 'updated_at': '2021-05-20T04:25:19Z', 'revision_number': 4, 'project_id': '39febf983bf24778a300f287dbeca829', 'subnets': [{'id': '49523a89-260b-4732-abff-507d74fed4c4', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:20.349424 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e80b2d45-e599-4744-bccb-18e57560b0ff None None] The sync data for ids ['5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'] in dv-cinder-782405-9-iscsi is: [{'id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'name': 'tempest-TestShelveInstance-251001111-router', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100'}], 'enable_snat': True}, 'gw_port_id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:08Z', 'updated_at': '2021-05-20T04:25:15Z', 'revision_number': 4, 'project_id': '39febf983bf24778a300f287dbeca829', 'gw_port': {'id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:d3:05:8a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:10Z', 'updated_at': '2021-05-20T04:25:16Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'c7df749b-cc66-4529-a666-5c9e0a997ff8', 'name': '', 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'mac_address': 'fa:16:3e:8f:27:b0', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_interface', 'standard_attr_id': 355, 'fixed_ips': [{'subnet_id': '49523a89-260b-4732-abff-507d74fed4c4', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:14Z', 'updated_at': '2021-05-20T04:25:19Z', 'revision_number': 4, 'project_id': '39febf983bf24778a300f287dbeca829', 'subnets': [{'id': '49523a89-260b-4732-abff-507d74fed4c4', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:25:20.448686 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:25:20.463571 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Sync routers for ids ['ecb83b2f-735e-4418-8208-a3dfb9673052'] in dv-cinder-782405-9-iscsi {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:25:20.477034 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Routers {'ecb83b2f-735e-4418-8208-a3dfb9673052'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:25:20.477707 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 6, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 6, "log_agent_heartbeats": false, "routers": 6},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:25:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['ecb83b2f-735e-4418-8208-a3dfb9673052'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:20.483271 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Notify callbacks [] for router, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:20.486941 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] 10.222.0.39 "POST /v2.0/routers HTTP/1.1" status: 201 len: 921 time: 2.4887352 May 20 04:25:20.489970 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 37764) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:20.499313 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-63caddb7-6e4a-4955-95d3-795a5cf3b2af req-04a1731a-dbc6-4ed5-9eb6-20532f04ed36 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:20.515642 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-63caddb7-6e4a-4955-95d3-795a5cf3b2af req-04a1731a-dbc6-4ed5-9eb6-20532f04ed36 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1834216 May 20 04:25:20.655491 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-63caddb7-6e4a-4955-95d3-795a5cf3b2af req-caefe071-06d2-4b32-98b7-7549fc5b1204 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1388412 May 20 04:25:20.746953 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3ba2ce05-98b8-45ca-899d-5c352fc80ee1 req-f4ffd6e3-77c3-488d-820f-75bb64843b3c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:20.755110 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3ba2ce05-98b8-45ca-899d-5c352fc80ee1 req-f4ffd6e3-77c3-488d-820f-75bb64843b3c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 945 time: 0.1709640 May 20 04:25:20.813082 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3ba2ce05-98b8-45ca-899d-5c352fc80ee1 req-532a33b6-036f-4df4-8b81-f37baf2eab61 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0567667 May 20 04:25:20.907990 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Set router ecb83b2f-735e-4418-8208-a3dfb9673052 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:25:21.329928 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'ecb83b2f-735e-4418-8208-a3dfb9673052', 'name': 'tempest-TestShelveInstance-2138875911-router', 'tenant_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.116'}], 'enable_snat': True}, 'gw_port_id': 'b57cd7ef-aeba-4bab-8870-543c319b914b', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:18Z', 'updated_at': '2021-05-20T04:25:19Z', 'revision_number': 3, 'project_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'gw_port': {'id': 'b57cd7ef-aeba-4bab-8870-543c319b914b', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:f5:44:9c', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'ecb83b2f-735e-4418-8208-a3dfb9673052', 'device_owner': 'network:router_gateway', 'standard_attr_id': 366, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.116', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:18Z', 'updated_at': '2021-05-20T04:25:18Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:21.334513 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Checking router: ecb83b2f-735e-4418-8208-a3dfb9673052 for host: dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:25:21.334867 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Updating router ecb83b2f-735e-4418-8208-a3dfb9673052 port b57cd7ef-aeba-4bab-8870-543c319b914b binding host dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 20 04:25:21.441416 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:21.523697 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-a99a2b7d-4249-4b54-ab72-2023007c98ad None None] Network c83ca109-2f8b-40c8-b2dd-b471ffdb582f requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:25:21.643580 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-82c01ea1-9c60-49f1-b63d-a037d27e9cd6 None None] Provisioning for port 424ccab3-7db4-445a-bbbc-4f034b243ce3 completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:25:21.662762 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:21.897065 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Notify callbacks [] for port, precommit_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:21.923058 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.provisioning_blocks [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Transition to ACTIVE for port object b57cd7ef-aeba-4bab-8870-543c319b914b will not be triggered until provisioned by entity L2. {{(pid=91646) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:25:21.936284 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:21.944689 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-148bf171-b3de-45b9-9dac-599a88dbdb32 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:21.960175 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 424ccab3-7db4-445a-bbbc-4f034b243ce3 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:25:21.968467 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c1b30715-da34-4eeb-920a-bb8e27d6d343 req-f5d0f933-cf71-47ac-a2fa-d400a89776ec tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:21.970029 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c1b30715-da34-4eeb-920a-bb8e27d6d343 req-f5d0f933-cf71-47ac-a2fa-d400a89776ec tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1376073 May 20 04:25:22.032318 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c1b30715-da34-4eeb-920a-bb8e27d6d343 req-8df6a78d-5f3d-46eb-8150-bb8a63924688 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0561252 May 20 04:25:22.064085 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-99d2e49f-b8e0-416e-933e-81c2f7b48574 None None] Pushing event updated for resources: {'Port': ['ID=b57cd7ef-aeba-4bab-8870-543c319b914b,revision_number=2']} {{(pid=91646) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:22.070339 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-99d2e49f-b8e0-416e-933e-81c2f7b48574 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.125s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:22.072919 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port 424ccab3-7db4-445a-bbbc-4f034b243ce3 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:25:22.091057 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Attempting to bind port b57cd7ef-aeba-4bab-8870-543c319b914b on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:25:22.091454 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Attempting to bind port b57cd7ef-aeba-4bab-8870-543c319b914b by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}] {{(pid=91646) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:25:22.092120 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Attempting to bind port b57cd7ef-aeba-4bab-8870-543c319b914b on network 712ce8c4-8feb-4719-bf13-e06992125cd9 {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:25:22.092807 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 424ccab3-7db4-445a-bbbc-4f034b243ce3 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:22.093186 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:22.098294 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 25, 15), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:25:22.098816 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Checking segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91646) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:25:22.099498 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:22.099614 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Bound using segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:25:22.100063 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Bound port: b57cd7ef-aeba-4bab-8870-543c319b914b, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}}] {{(pid=91646) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:25:22.100428 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:22.102288 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Allocated IP 172.20.0.1 (c83ca109-2f8b-40c8-b2dd-b471ffdb582f/747296cc-d241-444d-9f89-67006bc5546d/37ba7095-db27-4d59-9431-f8647c89ac1a) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:25:22.180472 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.db [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] For port b57cd7ef-aeba-4bab-8870-543c319b914b, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91646) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:25:22.184496 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.db [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] For port b57cd7ef-aeba-4bab-8870-543c319b914b, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=b57cd7ef-aeba-4bab-8870-543c319b914b,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91646) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:25:22.189663 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:22.195375 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-c393b33f-2a81-475d-994b-533acbff3d2f None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:22.253390 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:22.265439 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Routers {'ecb83b2f-735e-4418-8208-a3dfb9673052'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:25:22.266146 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 6, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 6, "log_agent_heartbeats": false, "routers": 6},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:25:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['ecb83b2f-735e-4418-8208-a3dfb9673052'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:22.296619 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-ff670637-4de7-4574-99d9-e53cb4a8d008 None None] Provisioning complete for port 424ccab3-7db4-445a-bbbc-4f034b243ce3 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:22.297004 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-ff670637-4de7-4574-99d9-e53cb4a8d008 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:22.308122 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:22.326656 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Transition to ACTIVE for port object 37ba7095-db27-4d59-9431-f8647c89ac1a will not be triggered until provisioned by entity DHCP. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:25:22.332250 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port 424ccab3-7db4-445a-bbbc-4f034b243ce3, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=424ccab3-7db4-445a-bbbc-4f034b243ce3,segment=NetworkSegment(6822d26b-245e-4501-a6f1-7f56846f560c),segment_id=6822d26b-245e-4501-a6f1-7f56846f560c)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:25:22.343955 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:22.344480 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device c7df749b-cc66-4529-a666-5c9e0a997ff8 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:25:22.352232 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-99d2e49f-b8e0-416e-933e-81c2f7b48574 None None] Pushing event updated for resources: {'Port': ['ID=b57cd7ef-aeba-4bab-8870-543c319b914b,revision_number=3']} {{(pid=91646) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:22.355340 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-99d2e49f-b8e0-416e-933e-81c2f7b48574 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.160s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:22.360279 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:22.366125 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:22.376101 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:22.428886 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Network c83ca109-2f8b-40c8-b2dd-b471ffdb582f is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:25:22.431980 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:25:22.435951 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:25:22.465565 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] Set router ecb83b2f-735e-4418-8208-a3dfb9673052 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:25:22.473054 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=424ccab3-7db4-445a-bbbc-4f034b243ce3,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:22.477749 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.102s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:22.517511 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Pushing event updated for resources: {'Port': ['ID=37ba7095-db27-4d59-9431-f8647c89ac1a,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:22.521799 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.156s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:22.542028 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-8003132'] for router_interface, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:22.580722 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port c7df749b-cc66-4529-a666-5c9e0a997ff8 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:25:22.610593 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port c7df749b-cc66-4529-a666-5c9e0a997ff8 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:22.610973 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:22.738034 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:22.767531 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:22.799568 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-9c03db08-6a35-4c22-a2f5-8b86a67f54ea None None] Provisioning for port 37ba7095-db27-4d59-9431-f8647c89ac1a completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:25:22.809712 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-9c03db08-6a35-4c22-a2f5-8b86a67f54ea None None] Provisioning complete for port 37ba7095-db27-4d59-9431-f8647c89ac1a triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:22.810092 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-9c03db08-6a35-4c22-a2f5-8b86a67f54ea None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:22.857749 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port c7df749b-cc66-4529-a666-5c9e0a997ff8, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=c7df749b-cc66-4529-a666-5c9e0a997ff8,segment=NetworkSegment(c7d77442-5ac6-48cf-ab59-3f5962fc1e3e),segment_id=c7d77442-5ac6-48cf-ab59-3f5962fc1e3e)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:25:22.866038 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:22.872852 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-a14e1d56-4a2f-40ff-aa8a-2790ce9d2ba7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:22.882842 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'ecb83b2f-735e-4418-8208-a3dfb9673052', 'name': 'tempest-TestShelveInstance-2138875911-router', 'tenant_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.116'}], 'enable_snat': True}, 'gw_port_id': 'b57cd7ef-aeba-4bab-8870-543c319b914b', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:18Z', 'updated_at': '2021-05-20T04:25:19Z', 'revision_number': 3, 'project_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'gw_port': {'id': 'b57cd7ef-aeba-4bab-8870-543c319b914b', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:f5:44:9c', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'ecb83b2f-735e-4418-8208-a3dfb9673052', 'device_owner': 'network:router_gateway', 'standard_attr_id': 366, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.116', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:18Z', 'updated_at': '2021-05-20T04:25:22Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:22.886222 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-dc301554-f39f-46a5-96e4-8f64b05849b5 None None] The sync data for ids ['ecb83b2f-735e-4418-8208-a3dfb9673052'] in dv-cinder-782405-9-iscsi is: [{'id': 'ecb83b2f-735e-4418-8208-a3dfb9673052', 'name': 'tempest-TestShelveInstance-2138875911-router', 'tenant_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.116'}], 'enable_snat': True}, 'gw_port_id': 'b57cd7ef-aeba-4bab-8870-543c319b914b', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:18Z', 'updated_at': '2021-05-20T04:25:19Z', 'revision_number': 3, 'project_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'gw_port': {'id': 'b57cd7ef-aeba-4bab-8870-543c319b914b', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:f5:44:9c', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'ecb83b2f-735e-4418-8208-a3dfb9673052', 'device_owner': 'network:router_gateway', 'standard_attr_id': 366, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.116', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:18Z', 'updated_at': '2021-05-20T04:25:22Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}] {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:25:22.915954 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-9c03db08-6a35-4c22-a2f5-8b86a67f54ea None None] Port 37ba7095-db27-4d59-9431-f8647c89ac1a cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:25:22.944608 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=c7df749b-cc66-4529-a666-5c9e0a997ff8,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:22.948153 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.075s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:22.987926 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:22.992512 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:23.016524 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-65344260-99ea-419c-9f68-fed55e74a4c4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:23.105551 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-7924897', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-8003163'] for router_interface, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:23.109243 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Pushing event updated for resources: {'Port': ['ID=37ba7095-db27-4d59-9431-f8647c89ac1a,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:23.112545 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:23.271122 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cb6d7273-123d-4dd0-a609-c80b4b4192b4 req-a5249979-c781-4262-8bc5-b1bdf2382606 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:23.272652 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cb6d7273-123d-4dd0-a609-c80b4b4192b4 req-a5249979-c781-4262-8bc5-b1bdf2382606 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1042500 May 20 04:25:23.296021 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Network c83ca109-2f8b-40c8-b2dd-b471ffdb582f is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:25:23.297356 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:25:23.339839 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cb6d7273-123d-4dd0-a609-c80b4b4192b4 req-da5e8a50-1caa-436b-86f2-0cdf7f488e6f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0612361 May 20 04:25:23.538860 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-46fe4308-e859-4bd2-8198-ee3ad64a71b1 None None] Provisioning complete for port 37ba7095-db27-4d59-9431-f8647c89ac1a triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:23.539383 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-46fe4308-e859-4bd2-8198-ee3ad64a71b1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:23.575373 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Router ecb83b2f-735e-4418-8208-a3dfb9673052 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91644) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:25:23.594211 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-46fe4308-e859-4bd2-8198-ee3ad64a71b1 None None] Port 37ba7095-db27-4d59-9431-f8647c89ac1a cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:25:23.767019 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:25:23.776327 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] 10.222.0.39 "PUT /v2.0/routers/ecb83b2f-735e-4418-8208-a3dfb9673052/add_router_interface HTTP/1.1" status: 200 len: 503 time: 3.2849619 May 20 04:25:23.906967 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device c7df749b-cc66-4529-a666-5c9e0a997ff8 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:25:23.979521 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port c7df749b-cc66-4529-a666-5c9e0a997ff8 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:23.979929 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:24.114746 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 424ccab3-7db4-445a-bbbc-4f034b243ce3 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:25:24.186897 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 424ccab3-7db4-445a-bbbc-4f034b243ce3 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:24.187308 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:24.324789 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device b57cd7ef-aeba-4bab-8870-543c319b914b up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:25:24.410550 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port b57cd7ef-aeba-4bab-8870-543c319b914b completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:25:24.425539 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port b57cd7ef-aeba-4bab-8870-543c319b914b triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:24.426074 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:24.594250 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-034a6427-c060-4c31-b807-9510f1bdb1db req-6504eb4d-704c-48a6-9aa8-178deddf1dac tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:24.599826 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-034a6427-c060-4c31-b807-9510f1bdb1db req-6504eb4d-704c-48a6-9aa8-178deddf1dac tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0967796 May 20 04:25:24.612631 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:24.629285 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port b57cd7ef-aeba-4bab-8870-543c319b914b, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=b57cd7ef-aeba-4bab-8870-543c319b914b,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:25:24.633395 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:24.635178 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:24.649097 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-034a6427-c060-4c31-b807-9510f1bdb1db req-96ba9aa8-42fd-4e54-8878-cd9240f65753 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0464611 May 20 04:25:24.712962 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=b57cd7ef-aeba-4bab-8870-543c319b914b,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:24.716212 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Sync routers for ids ['ecb83b2f-735e-4418-8208-a3dfb9673052'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:25:24.722173 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.087s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:24.726733 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Routers {'ecb83b2f-735e-4418-8208-a3dfb9673052'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:25:24.728575 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 6, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 6, "log_agent_heartbeats": false, "routers": 6},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:25:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['ecb83b2f-735e-4418-8208-a3dfb9673052'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:24.880262 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 37822) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:24.886780 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-735b54fd-fca9-4a2c-87d6-a329e70019b5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Request body: {'security_group': {'name': 'tempest-secgroup-smoke-434830226', 'description': 'tempest-secgroup-smoke-434830226 description', 'project_id': '39febf983bf24778a300f287dbeca829'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:24.892725 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-735b54fd-fca9-4a2c-87d6-a329e70019b5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:24.896571 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-735b54fd-fca9-4a2c-87d6-a329e70019b5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Usage tracker for resource:security_group and tenant:39febf983bf24778a300f287dbeca829 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:25:24.900075 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-735b54fd-fca9-4a2c-87d6-a329e70019b5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:25:24.905719 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-735b54fd-fca9-4a2c-87d6-a329e70019b5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:24.912023 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Set router ecb83b2f-735e-4418-8208-a3dfb9673052 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:25:24.916816 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-735b54fd-fca9-4a2c-87d6-a329e70019b5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:24.936892 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-735b54fd-fca9-4a2c-87d6-a329e70019b5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:24.940947 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-735b54fd-fca9-4a2c-87d6-a329e70019b5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Usage tracker for resource:security_group_rule and tenant:39febf983bf24778a300f287dbeca829 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:25:24.943831 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-735b54fd-fca9-4a2c-87d6-a329e70019b5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:25:24.949568 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-735b54fd-fca9-4a2c-87d6-a329e70019b5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:25.021190 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-735b54fd-fca9-4a2c-87d6-a329e70019b5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.027266 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-735b54fd-fca9-4a2c-87d6-a329e70019b5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.033181 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-8d161969-b827-4f57-ba13-0474ce3ebbb9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:25.072524 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-735b54fd-fca9-4a2c-87d6-a329e70019b5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Pushing event updated for resources: {'SecurityGroup': ['ID=9eacb756-312a-4612-a34b-e9da9dda4f71,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:25.076273 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-735b54fd-fca9-4a2c-87d6-a329e70019b5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:25.077859 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-735b54fd-fca9-4a2c-87d6-a329e70019b5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.079267 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-735b54fd-fca9-4a2c-87d6-a329e70019b5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1792 time: 0.1971803 May 20 04:25:25.082534 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 37826) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:25.087854 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-0a67906b-4a1e-4c6c-9b71-28bd8605a232 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Request body: {'security_group_rule': {'security_group_id': '9eacb756-312a-4612-a34b-e9da9dda4f71', 'project_id': '39febf983bf24778a300f287dbeca829', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:25.093976 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-0a67906b-4a1e-4c6c-9b71-28bd8605a232 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:25.097881 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-0a67906b-4a1e-4c6c-9b71-28bd8605a232 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Usage tracker for resource:security_group_rule and tenant:39febf983bf24778a300f287dbeca829 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:25:25.100742 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-0a67906b-4a1e-4c6c-9b71-28bd8605a232 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:25:25.107863 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-0a67906b-4a1e-4c6c-9b71-28bd8605a232 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:25.125074 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-0a67906b-4a1e-4c6c-9b71-28bd8605a232 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.189053 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-0a67906b-4a1e-4c6c-9b71-28bd8605a232 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.193095 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-0a67906b-4a1e-4c6c-9b71-28bd8605a232 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.195289 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:25.217564 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0a67906b-4a1e-4c6c-9b71-28bd8605a232 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=225ac7d1-d435-4283-b051-20fe784f5c7d,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:25.221763 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-0a67906b-4a1e-4c6c-9b71-28bd8605a232 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:25.226587 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-0a67906b-4a1e-4c6c-9b71-28bd8605a232 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.227767 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-0a67906b-4a1e-4c6c-9b71-28bd8605a232 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.1441026 May 20 04:25:25.230437 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 37830) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:25.235651 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-0c288c8f-ede7-4c66-af13-77c389c06be8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Request body: {'security_group_rule': {'security_group_id': '9eacb756-312a-4612-a34b-e9da9dda4f71', 'project_id': '39febf983bf24778a300f287dbeca829', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'egress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:25.240775 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'ecb83b2f-735e-4418-8208-a3dfb9673052', 'name': 'tempest-TestShelveInstance-2138875911-router', 'tenant_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.116'}], 'enable_snat': True}, 'gw_port_id': 'b57cd7ef-aeba-4bab-8870-543c319b914b', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:18Z', 'updated_at': '2021-05-20T04:25:22Z', 'revision_number': 4, 'project_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'gw_port': {'id': 'b57cd7ef-aeba-4bab-8870-543c319b914b', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:f5:44:9c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ecb83b2f-735e-4418-8208-a3dfb9673052', 'device_owner': 'network:router_gateway', 'standard_attr_id': 366, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.116', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:18Z', 'updated_at': '2021-05-20T04:25:24Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '37ba7095-db27-4d59-9431-f8647c89ac1a', 'name': '', 'network_id': 'c83ca109-2f8b-40c8-b2dd-b471ffdb582f', 'tenant_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'mac_address': 'fa:16:3e:27:c0:77', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'ecb83b2f-735e-4418-8208-a3dfb9673052', 'device_owner': 'network:router_interface', 'standard_attr_id': 367, 'fixed_ips': [{'subnet_id': '747296cc-d241-444d-9f89-67006bc5546d', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:21Z', 'updated_at': '2021-05-20T04:25:22Z', 'revision_number': 2, 'project_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'subnets': [{'id': '747296cc-d241-444d-9f89-67006bc5546d', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:25.241131 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-0c288c8f-ede7-4c66-af13-77c389c06be8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:25.244637 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Checking router: ecb83b2f-735e-4418-8208-a3dfb9673052 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:25:25.244929 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-0c288c8f-ede7-4c66-af13-77c389c06be8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Usage tracker for resource:security_group_rule and tenant:39febf983bf24778a300f287dbeca829 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:25:25.245101 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Updating router ecb83b2f-735e-4418-8208-a3dfb9673052 port 37ba7095-db27-4d59-9431-f8647c89ac1a binding host dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 20 04:25:25.247826 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-0c288c8f-ede7-4c66-af13-77c389c06be8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:25:25.253721 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-0c288c8f-ede7-4c66-af13-77c389c06be8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:25.269508 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-0c288c8f-ede7-4c66-af13-77c389c06be8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.298366 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.336589 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-0c288c8f-ede7-4c66-af13-77c389c06be8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.340307 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-0c288c8f-ede7-4c66-af13-77c389c06be8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.342602 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:25.364136 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0c288c8f-ede7-4c66-af13-77c389c06be8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=30397de1-1da8-4042-9298-c184e1fa3077,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:25.366565 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-0c288c8f-ede7-4c66-af13-77c389c06be8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.024s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:25.374224 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-0c288c8f-ede7-4c66-af13-77c389c06be8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.375511 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-0c288c8f-ede7-4c66-af13-77c389c06be8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 763 time: 0.1441362 May 20 04:25:25.378405 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 37834) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:25.383498 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-fe34eebb-eca6-48c8-906f-e505e5188787 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Request body: {'security_group_rule': {'security_group_id': '9eacb756-312a-4612-a34b-e9da9dda4f71', 'project_id': '39febf983bf24778a300f287dbeca829', 'protocol': 'icmp', 'direction': 'ingress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:25.389158 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-fe34eebb-eca6-48c8-906f-e505e5188787 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:25.401969 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-fe34eebb-eca6-48c8-906f-e505e5188787 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Usage tracker for resource:security_group_rule and tenant:39febf983bf24778a300f287dbeca829 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:25:25.404977 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-fe34eebb-eca6-48c8-906f-e505e5188787 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Quota usage for security_group_rule was recalculated. Used quota:8. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:25:25.411286 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-fe34eebb-eca6-48c8-906f-e505e5188787 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 8; quota limit: 100; headroom:92 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:25.431091 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-fe34eebb-eca6-48c8-906f-e505e5188787 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.496177 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-fe34eebb-eca6-48c8-906f-e505e5188787 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.500695 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-fe34eebb-eca6-48c8-906f-e505e5188787 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.508833 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:25.536869 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fe34eebb-eca6-48c8-906f-e505e5188787 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=ee82f9ed-55d7-436d-b22a-c8586cfea4fa,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:25.541105 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-fe34eebb-eca6-48c8-906f-e505e5188787 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.032s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:25.544299 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-fe34eebb-eca6-48c8-906f-e505e5188787 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.545563 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-fe34eebb-eca6-48c8-906f-e505e5188787 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.1662745 May 20 04:25:25.548499 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 37842) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:25.553837 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-36c4c267-7465-4370-9d75-02f5af702bd8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Request body: {'security_group_rule': {'security_group_id': '9eacb756-312a-4612-a34b-e9da9dda4f71', 'project_id': '39febf983bf24778a300f287dbeca829', 'protocol': 'icmp', 'direction': 'egress'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:25.559737 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-36c4c267-7465-4370-9d75-02f5af702bd8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:25.563695 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-36c4c267-7465-4370-9d75-02f5af702bd8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Usage tracker for resource:security_group_rule and tenant:39febf983bf24778a300f287dbeca829 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:25:25.566735 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-36c4c267-7465-4370-9d75-02f5af702bd8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Quota usage for security_group_rule was recalculated. Used quota:9. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:25:25.573655 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-36c4c267-7465-4370-9d75-02f5af702bd8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 9; quota limit: 100; headroom:91 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:25.590453 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-36c4c267-7465-4370-9d75-02f5af702bd8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.652582 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-36c4c267-7465-4370-9d75-02f5af702bd8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.657807 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-36c4c267-7465-4370-9d75-02f5af702bd8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.659937 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-dae9e12b-40d6-4925-bc03-edf01ea3b3cb tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:25.662711 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Notify callbacks [] for port, precommit_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.675429 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Transition to ACTIVE for port object 37ba7095-db27-4d59-9431-f8647c89ac1a will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:25:25.681988 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.685165 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-36c4c267-7465-4370-9d75-02f5af702bd8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=efff347b-a107-4c4d-8a92-e96262bae7ff,revision_number=0']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:25.686356 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:25.696208 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-36c4c267-7465-4370-9d75-02f5af702bd8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.036s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:25.701338 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-36c4c267-7465-4370-9d75-02f5af702bd8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.705442 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-36c4c267-7465-4370-9d75-02f5af702bd8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 768 time: 0.1534061 May 20 04:25:25.706036 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 37848) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:25.711601 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-8e4f3d6c-879a-4ad6-8e74-67a00492065b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Request body: {'security_group_rule': {'security_group_id': '9eacb756-312a-4612-a34b-e9da9dda4f71', 'project_id': '39febf983bf24778a300f287dbeca829', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'ingress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:25.718656 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-8e4f3d6c-879a-4ad6-8e74-67a00492065b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:25.727494 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-8e4f3d6c-879a-4ad6-8e74-67a00492065b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Usage tracker for resource:security_group_rule and tenant:39febf983bf24778a300f287dbeca829 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:25:25.731468 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-8e4f3d6c-879a-4ad6-8e74-67a00492065b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Quota usage for security_group_rule was recalculated. Used quota:10. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:25:25.740913 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-8e4f3d6c-879a-4ad6-8e74-67a00492065b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 10; quota limit: 100; headroom:90 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:25.766356 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.securitygroups_db [None req-8e4f3d6c-879a-4ad6-8e74-67a00492065b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Project 39febf983bf24778a300f287dbeca829 added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 20 04:25:25.821895 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8e4f3d6c-879a-4ad6-8e74-67a00492065b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.874426 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device b57cd7ef-aeba-4bab-8870-543c319b914b up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:25:25.932521 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2f36aeb0-0bb0-47ca-85f5-ff1a8a93c5f9 req-84be4045-b7eb-4a14-adf1-eddb73394e59 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:25.943390 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2f36aeb0-0bb0-47ca-85f5-ff1a8a93c5f9 req-84be4045-b7eb-4a14-adf1-eddb73394e59 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1788313 May 20 04:25:25.947143 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dfc6f6f8-a39a-4ab9-a478-4083e76fe6c7 None None] Pushing event updated for resources: {'Port': ['ID=37ba7095-db27-4d59-9431-f8647c89ac1a,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:25.953186 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-dfc6f6f8-a39a-4ab9-a478-4083e76fe6c7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.267s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:25.966206 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Attempting to bind port 37ba7095-db27-4d59-9431-f8647c89ac1a on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:25:25.966574 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Attempting to bind port 37ba7095-db27-4d59-9431-f8647c89ac1a by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '6822d26b-245e-4501-a6f1-7f56846f560c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 840, 'network_id': 'c83ca109-2f8b-40c8-b2dd-b471ffdb582f'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:25:25.967620 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Attempting to bind port 37ba7095-db27-4d59-9431-f8647c89ac1a on network c83ca109-2f8b-40c8-b2dd-b471ffdb582f {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:25:25.976503 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 25, 15), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:25:25.976874 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Checking segment: {'id': '6822d26b-245e-4501-a6f1-7f56846f560c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 840, 'network_id': 'c83ca109-2f8b-40c8-b2dd-b471ffdb582f'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:25:25.977534 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:25.977929 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Bound using segment: {'id': '6822d26b-245e-4501-a6f1-7f56846f560c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 840, 'network_id': 'c83ca109-2f8b-40c8-b2dd-b471ffdb582f'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:25:25.978478 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Bound port: 37ba7095-db27-4d59-9431-f8647c89ac1a, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '6822d26b-245e-4501-a6f1-7f56846f560c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 840, 'network_id': 'c83ca109-2f8b-40c8-b2dd-b471ffdb582f'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:25:25.978790 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:26.007101 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8e4f3d6c-879a-4ad6-8e74-67a00492065b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:26.017481 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2f36aeb0-0bb0-47ca-85f5-ff1a8a93c5f9 req-f4b07bfc-e0d2-4716-aa12-62be05d1fa3b tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0644457 May 20 04:25:26.018715 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8e4f3d6c-879a-4ad6-8e74-67a00492065b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:26.020800 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-33908b5a-4e53-4990-91b8-22e0d17bbac8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:26.049954 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8e4f3d6c-879a-4ad6-8e74-67a00492065b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=44e3c5ef-cfa0-4130-b7d9-c441f1d9909e,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:26.053994 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-8e4f3d6c-879a-4ad6-8e74-67a00492065b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.033s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:26.064639 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8e4f3d6c-879a-4ad6-8e74-67a00492065b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:26.066237 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-8e4f3d6c-879a-4ad6-8e74-67a00492065b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 774 time: 0.3590078 May 20 04:25:26.069546 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 37858) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:26.074911 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-9e7210a2-e0bb-4a3d-865a-7bb3a092f6b0 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Request body: {'security_group_rule': {'security_group_id': '9eacb756-312a-4612-a34b-e9da9dda4f71', 'project_id': '39febf983bf24778a300f287dbeca829', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'egress'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:26.075413 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port b57cd7ef-aeba-4bab-8870-543c319b914b triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:26.080278 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:26.081825 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-9e7210a2-e0bb-4a3d-865a-7bb3a092f6b0 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:26.085872 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-9e7210a2-e0bb-4a3d-865a-7bb3a092f6b0 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Usage tracker for resource:security_group_rule and tenant:39febf983bf24778a300f287dbeca829 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:25:26.089056 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-9e7210a2-e0bb-4a3d-865a-7bb3a092f6b0 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Quota usage for security_group_rule was recalculated. Used quota:11. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:25:26.095356 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-9e7210a2-e0bb-4a3d-865a-7bb3a092f6b0 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 11; quota limit: 100; headroom:89 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:26.108040 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.db.securitygroups_db [None req-9e7210a2-e0bb-4a3d-865a-7bb3a092f6b0 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Project 39febf983bf24778a300f287dbeca829 added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 20 04:25:26.114374 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9e7210a2-e0bb-4a3d-865a-7bb3a092f6b0 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:26.192118 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9e7210a2-e0bb-4a3d-865a-7bb3a092f6b0 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:26.200625 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9e7210a2-e0bb-4a3d-865a-7bb3a092f6b0 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:26.202432 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] For port 37ba7095-db27-4d59-9431-f8647c89ac1a, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:25:26.208819 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] For port 37ba7095-db27-4d59-9431-f8647c89ac1a, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=37ba7095-db27-4d59-9431-f8647c89ac1a,segment=NetworkSegment(6822d26b-245e-4501-a6f1-7f56846f560c),segment_id=6822d26b-245e-4501-a6f1-7f56846f560c)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:25:26.214496 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:26.217145 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:26.221053 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-e730247e-f308-4816-8b4d-1425b012cfe5 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:26.233563 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9e7210a2-e0bb-4a3d-865a-7bb3a092f6b0 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=1e4b2f9c-229a-4300-8672-142eb94b95a7,revision_number=0']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:26.239957 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-9e7210a2-e0bb-4a3d-865a-7bb3a092f6b0 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.025s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:26.244683 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9e7210a2-e0bb-4a3d-865a-7bb3a092f6b0 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:26.254043 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-9e7210a2-e0bb-4a3d-865a-7bb3a092f6b0 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 773 time: 0.1771145 May 20 04:25:26.351614 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Routers {'ecb83b2f-735e-4418-8208-a3dfb9673052'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:25:26.352422 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 6, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 6, "log_agent_heartbeats": false, "routers": 6},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:25:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['ecb83b2f-735e-4418-8208-a3dfb9673052'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:26.439151 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dfc6f6f8-a39a-4ab9-a478-4083e76fe6c7 None None] Pushing event updated for resources: {'Port': ['ID=37ba7095-db27-4d59-9431-f8647c89ac1a,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:26.442697 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-dfc6f6f8-a39a-4ab9-a478-4083e76fe6c7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.222s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:26.457477 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-d48790c9-9937-40ac-b113-448f6bf623d7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-195167850&fields=id&tenant_id=0e3baa0ea24d42f3ab59e40845b350fd HTTP/1.1" status: 200 len: 267 time: 0.0493131 May 20 04:25:26.622099 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] Set router ecb83b2f-735e-4418-8208-a3dfb9673052 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:25:26.657465 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-c5d82973-c55b-45ee-b493-1a2b3ed7f7ce tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:26.671027 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-c5d82973-c55b-45ee-b493-1a2b3ed7f7ce tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/networks?id=14c537db-01aa-4360-a348-47f94c75e03b HTTP/1.1" status: 200 len: 810 time: 0.2035711 May 20 04:25:26.680928 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-eec2e254-1c2f-4453-a58e-2f4d5bc9483f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/quotas/0e3baa0ea24d42f3ab59e40845b350fd HTTP/1.1" status: 200 len: 375 time: 0.0090609 May 20 04:25:26.768525 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-a127f19f-70e0-41e1-b7b4-1fa3a7a2a3a8 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=0e3baa0ea24d42f3ab59e40845b350fd&fields=id HTTP/1.1" status: 200 len: 302 time: 0.0803964 May 20 04:25:26.913153 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-3002db7a-a74b-462a-ac8f-b87ed5d267cb service neutron] 10.222.0.39 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1292472 May 20 04:25:26.952687 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'ecb83b2f-735e-4418-8208-a3dfb9673052', 'name': 'tempest-TestShelveInstance-2138875911-router', 'tenant_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.116'}], 'enable_snat': True}, 'gw_port_id': 'b57cd7ef-aeba-4bab-8870-543c319b914b', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:18Z', 'updated_at': '2021-05-20T04:25:22Z', 'revision_number': 4, 'project_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'gw_port': {'id': 'b57cd7ef-aeba-4bab-8870-543c319b914b', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:f5:44:9c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ecb83b2f-735e-4418-8208-a3dfb9673052', 'device_owner': 'network:router_gateway', 'standard_attr_id': 366, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.116', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:18Z', 'updated_at': '2021-05-20T04:25:24Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '37ba7095-db27-4d59-9431-f8647c89ac1a', 'name': '', 'network_id': 'c83ca109-2f8b-40c8-b2dd-b471ffdb582f', 'tenant_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'mac_address': 'fa:16:3e:27:c0:77', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'ecb83b2f-735e-4418-8208-a3dfb9673052', 'device_owner': 'network:router_interface', 'standard_attr_id': 367, 'fixed_ips': [{'subnet_id': '747296cc-d241-444d-9f89-67006bc5546d', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:21Z', 'updated_at': '2021-05-20T04:25:26Z', 'revision_number': 4, 'project_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'subnets': [{'id': '747296cc-d241-444d-9f89-67006bc5546d', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:26.958460 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7ba3c537-75f8-4161-9150-3644fa80e802 None None] The sync data for ids ['ecb83b2f-735e-4418-8208-a3dfb9673052'] in dv-cinder-782405-9-iscsi is: [{'id': 'ecb83b2f-735e-4418-8208-a3dfb9673052', 'name': 'tempest-TestShelveInstance-2138875911-router', 'tenant_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.116'}], 'enable_snat': True}, 'gw_port_id': 'b57cd7ef-aeba-4bab-8870-543c319b914b', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:18Z', 'updated_at': '2021-05-20T04:25:22Z', 'revision_number': 4, 'project_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'gw_port': {'id': 'b57cd7ef-aeba-4bab-8870-543c319b914b', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:f5:44:9c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ecb83b2f-735e-4418-8208-a3dfb9673052', 'device_owner': 'network:router_gateway', 'standard_attr_id': 366, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.116', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:18Z', 'updated_at': '2021-05-20T04:25:24Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '37ba7095-db27-4d59-9431-f8647c89ac1a', 'name': '', 'network_id': 'c83ca109-2f8b-40c8-b2dd-b471ffdb582f', 'tenant_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'mac_address': 'fa:16:3e:27:c0:77', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'ecb83b2f-735e-4418-8208-a3dfb9673052', 'device_owner': 'network:router_interface', 'standard_attr_id': 367, 'fixed_ips': [{'subnet_id': '747296cc-d241-444d-9f89-67006bc5546d', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:21Z', 'updated_at': '2021-05-20T04:25:26Z', 'revision_number': 4, 'project_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'subnets': [{'id': '747296cc-d241-444d-9f89-67006bc5546d', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:25:27.070468 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-d60812f2-7366-4693-81a6-6b719b55fad3 service neutron] 10.222.0.39 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1511369 May 20 04:25:27.373266 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-83293980-06dd-4b79-9e8e-e4f6fea89ba0 req-aa9c8c97-ffae-435a-8c70-a9ee5d4a98d1 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:27.375084 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-83293980-06dd-4b79-9e8e-e4f6fea89ba0 req-aa9c8c97-ffae-435a-8c70-a9ee5d4a98d1 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0929220 May 20 04:25:27.435080 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-83293980-06dd-4b79-9e8e-e4f6fea89ba0 req-60c5007c-ca85-4f2a-8baa-e5734998b8ac tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0519278 May 20 04:25:27.527910 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2f6e9176-8bc3-4346-bfc9-695a602057b1 req-7d058947-7522-47a4-ad13-0cbdd8044a8f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 210 time: 0.0483263 May 20 04:25:27.858704 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 37ba7095-db27-4d59-9431-f8647c89ac1a up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:25:27.934677 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port 37ba7095-db27-4d59-9431-f8647c89ac1a completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:25:27.945094 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 37ba7095-db27-4d59-9431-f8647c89ac1a triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:27.945583 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:28.075627 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:28.111066 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port 37ba7095-db27-4d59-9431-f8647c89ac1a, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=37ba7095-db27-4d59-9431-f8647c89ac1a,segment=NetworkSegment(6822d26b-245e-4501-a6f1-7f56846f560c),segment_id=6822d26b-245e-4501-a6f1-7f56846f560c)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:25:28.116480 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:28.118498 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:28.190891 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=37ba7095-db27-4d59-9431-f8647c89ac1a,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:28.194505 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.076s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:28.799056 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c3255e70-b9a0-4b83-994c-5050dde23056 req-d1e5ce7f-c094-4f7d-afcc-3cf1dc773966 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 210 time: 0.1500130 May 20 04:25:28.873027 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-de3ad29a-5e14-4633-ae8d-f15d5781c4cd req-48762e36-4dba-4b14-8cd9-24eae5649a72 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:28.882456 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-de3ad29a-5e14-4633-ae8d-f15d5781c4cd req-48762e36-4dba-4b14-8cd9-24eae5649a72 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1214013 May 20 04:25:28.886853 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-d9d21c39-2830-4eca-aada-0ddb5e441c52 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:28.887739 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-d9d21c39-2830-4eca-aada-0ddb5e441c52 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Synchronizing usage tracker for tenant:0e3baa0ea24d42f3ab59e40845b350fd on resource:network {{(pid=91644) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:25:28.908123 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-d9d21c39-2830-4eca-aada-0ddb5e441c52 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Unset dirty status for tenant:0e3baa0ea24d42f3ab59e40845b350fd on resource:network {{(pid=91644) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:25:28.909787 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-d9d21c39-2830-4eca-aada-0ddb5e441c52 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.74 "GET /v2.0/networks?id=14c537db-01aa-4360-a348-47f94c75e03b HTTP/1.1" status: 200 len: 810 time: 0.1807685 May 20 04:25:28.939478 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-de3ad29a-5e14-4633-ae8d-f15d5781c4cd req-1c418795-c976-4853-a9d8-b95efd345631 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0559957 May 20 04:25:28.976601 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-de2efaaf-d6ff-48e0-8deb-101dbf0647db tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Synchronizing usage tracker for tenant:0e3baa0ea24d42f3ab59e40845b350fd on resource:security_group {{(pid=91644) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:25:28.996086 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-de2efaaf-d6ff-48e0-8deb-101dbf0647db tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Unset dirty status for tenant:0e3baa0ea24d42f3ab59e40845b350fd on resource:security_group {{(pid=91644) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:25:28.997215 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-de2efaaf-d6ff-48e0-8deb-101dbf0647db tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.74 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=0e3baa0ea24d42f3ab59e40845b350fd HTTP/1.1" status: 200 len: 379 time: 0.0759439 May 20 04:25:29.014997 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-5820cc2c-3cdd-49e4-a270-24544662a402 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Request body: {'port': {'device_id': 'bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'admin_state_up': True, 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'security_groups': ['5a7943e2-185b-4112-9281-61ab71b1afeb']}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:29.158050 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-5820cc2c-3cdd-49e4-a270-24544662a402 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:29.162235 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-5820cc2c-3cdd-49e4-a270-24544662a402 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Usage tracker for resource:port and tenant:0e3baa0ea24d42f3ab59e40845b350fd is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:25:29.165191 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-5820cc2c-3cdd-49e4-a270-24544662a402 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Quota usage for port was recalculated. Used quota:2. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:25:29.171121 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-5820cc2c-3cdd-49e4-a270-24544662a402 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:29.185338 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-5820cc2c-3cdd-49e4-a270-24544662a402 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Made reservation on behalf of 0e3baa0ea24d42f3ab59e40845b350fd for: {'port': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:25:29.190935 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-5820cc2c-3cdd-49e4-a270-24544662a402 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:29.362466 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-5820cc2c-3cdd-49e4-a270-24544662a402 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Allocated IP 172.20.0.9 (14c537db-01aa-4360-a348-47f94c75e03b/cdea475c-2929-4ae4-9271-87cc2fb9a378/4a6aa217-4367-4b88-9233-0a8aeee2352d) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:25:29.553621 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-5820cc2c-3cdd-49e4-a270-24544662a402 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:29.570306 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-5820cc2c-3cdd-49e4-a270-24544662a402 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Transition to ACTIVE for port object 4a6aa217-4367-4b88-9233-0a8aeee2352d will not be triggered until provisioned by entity DHCP. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:25:29.597309 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-5820cc2c-3cdd-49e4-a270-24544662a402 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:29.602708 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:29.671138 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-5820cc2c-3cdd-49e4-a270-24544662a402 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Network 14c537db-01aa-4360-a348-47f94c75e03b is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:25:29.674095 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-5820cc2c-3cdd-49e4-a270-24544662a402 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:25:29.678050 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-5820cc2c-3cdd-49e4-a270-24544662a402 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:25:29.709389 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5820cc2c-3cdd-49e4-a270-24544662a402 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event updated for resources: {'Port': ['ID=4a6aa217-4367-4b88-9233-0a8aeee2352d,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:29.716516 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-5820cc2c-3cdd-49e4-a270-24544662a402 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:29.747144 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-5820cc2c-3cdd-49e4-a270-24544662a402 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:29.747878 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-5820cc2c-3cdd-49e4-a270-24544662a402 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:29.773341 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-5820cc2c-3cdd-49e4-a270-24544662a402 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 0.7631931 May 20 04:25:29.790090 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Request body: {'port': {'device_id': 'bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:29.802249 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 37ba7095-db27-4d59-9431-f8647c89ac1a up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:25:29.875022 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-e0909bb6-80ea-463f-972b-93e6a6d7c20d None None] Provisioning for port 4a6aa217-4367-4b88-9233-0a8aeee2352d completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:25:29.890511 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-e0909bb6-80ea-463f-972b-93e6a6d7c20d None None] Provisioning complete for port 4a6aa217-4367-4b88-9233-0a8aeee2352d triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:29.890889 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e0909bb6-80ea-463f-972b-93e6a6d7c20d None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:29.912582 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:29.936440 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 37ba7095-db27-4d59-9431-f8647c89ac1a triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:29.936823 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:29.997221 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-e0909bb6-80ea-463f-972b-93e6a6d7c20d None None] Port 4a6aa217-4367-4b88-9233-0a8aeee2352d cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:25:30.007853 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f2c2422d-ae7d-4fc1-97f6-b81f6860efa5 req-e059947a-7435-439f-9ad5-cc957bb4a70b tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:30.015763 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f2c2422d-ae7d-4fc1-97f6-b81f6860efa5 req-e059947a-7435-439f-9ad5-cc957bb4a70b tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 931 time: 0.0937927 May 20 04:25:30.079435 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f2c2422d-ae7d-4fc1-97f6-b81f6860efa5 req-d5eb6126-6803-4ad8-93b0-89c7888ed1b9 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0625226 May 20 04:25:30.209346 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-57b34bb5-4c70-461f-bf77-4153e4eb3d97 req-5520105c-af6f-4e34-a9b7-f039a29e8e04 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:30.210802 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-57b34bb5-4c70-461f-bf77-4153e4eb3d97 req-5520105c-af6f-4e34-a9b7-f039a29e8e04 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0936866 May 20 04:25:30.259040 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-57b34bb5-4c70-461f-bf77-4153e4eb3d97 req-bbd24982-81ad-495a-ae26-83bf7b1ca1a2 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0422547 May 20 04:25:30.413848 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 4a6aa217-4367-4b88-9233-0a8aeee2352d {{(pid=91644) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:25:30.421231 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:30.435110 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Transition to ACTIVE for port object 4a6aa217-4367-4b88-9233-0a8aeee2352d will not be triggered until provisioned by entity L2. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:25:30.439718 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:30.444166 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:30.594839 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Pushing event updated for resources: {'Port': ['ID=4a6aa217-4367-4b88-9233-0a8aeee2352d,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:30.602605 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.158s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:30.674665 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Attempting to bind port 4a6aa217-4367-4b88-9233-0a8aeee2352d on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:25:30.674992 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Attempting to bind port 4a6aa217-4367-4b88-9233-0a8aeee2352d by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:25:30.675346 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Attempting to bind port 4a6aa217-4367-4b88-9233-0a8aeee2352d on network 14c537db-01aa-4360-a348-47f94c75e03b {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:25:30.679978 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Port 4a6aa217-4367-4b88-9233-0a8aeee2352d on network 14c537db-01aa-4360-a348-47f94c75e03b not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:25:30.680305 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Attempting to bind port 4a6aa217-4367-4b88-9233-0a8aeee2352d on network 14c537db-01aa-4360-a348-47f94c75e03b {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:25:30.685424 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 25, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:25:30.685749 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Checking segment: {'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:25:30.686117 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Bound using segment: {'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:25:30.686527 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Bound port: 4a6aa217-4367-4b88-9233-0a8aeee2352d, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'}}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:25:30.686854 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:30.780171 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] For port 4a6aa217-4367-4b88-9233-0a8aeee2352d, host hv-cinder-78240, cleared binding levels {{(pid=91644) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:25:30.783139 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] For port 4a6aa217-4367-4b88-9233-0a8aeee2352d, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=4a6aa217-4367-4b88-9233-0a8aeee2352d,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91644) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:25:30.788730 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:30.794983 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:30.852405 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:30.853017 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:30.854395 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] 10.222.0.74 "PUT /v2.0/ports/4a6aa217-4367-4b88-9233-0a8aeee2352d HTTP/1.1" status: 200 len: 1093 time: 1.0687983 May 20 04:25:30.854870 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71', 'tag': '4a6aa217-4367-4b88-9233-0a8aeee2352d'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:25:30.882660 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Pushing event updated for resources: {'Port': ['ID=4a6aa217-4367-4b88-9233-0a8aeee2352d,revision_number=3']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:30.898323 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.095s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:30.908544 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71', 'tag': '4a6aa217-4367-4b88-9233-0a8aeee2352d', 'status': 'completed', 'code': 200} May 20 04:25:30.994806 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-78aa675f-5cd9-4964-9adf-7d65c5be09ed service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:30.996529 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-78aa675f-5cd9-4964-9adf-7d65c5be09ed service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=0e3baa0ea24d42f3ab59e40845b350fd&device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 1096 time: 0.0765660 May 20 04:25:31.075839 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-65995fc7-ec52-4c64-a6a4-5870700f5a21 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=4a6aa217-4367-4b88-9233-0a8aeee2352d HTTP/1.1" status: 200 len: 217 time: 0.0620174 May 20 04:25:31.150655 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-e9c8dafd-f6b3-4b5e-8624-c7a010ca5c8e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:31.152128 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-e9c8dafd-f6b3-4b5e-8624-c7a010ca5c8e service neutron] 10.222.0.74 "GET /v2.0/subnets?id=cdea475c-2929-4ae4-9271-87cc2fb9a378 HTTP/1.1" status: 200 len: 849 time: 0.0684855 May 20 04:25:31.242105 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-8f645821-dc92-4227-a5a0-a17a89304807 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:31.243753 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-8f645821-dc92-4227-a5a0-a17a89304807 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=14c537db-01aa-4360-a348-47f94c75e03b&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0790782 May 20 04:25:31.268025 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-db8201be-a1d3-4d05-b80c-b85ee555591e req-37c1a80a-40c8-4567-b6c8-153cbbb770c8 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:31.269454 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-db8201be-a1d3-4d05-b80c-b85ee555591e req-37c1a80a-40c8-4567-b6c8-153cbbb770c8 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 943 time: 0.0836508 May 20 04:25:31.321817 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-db8201be-a1d3-4d05-b80c-b85ee555591e req-801314c5-fb3d-4c85-bd6a-ea18593b9848 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0425427 May 20 04:25:31.389237 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-437f3924-b93b-401d-9195-898aef905819 service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1373630 May 20 04:25:31.511950 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a63962c7-f584-400c-894a-5c7bba374d68 req-51cfb162-36fa-4b7d-82c6-2c4d0729e0e0 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:31.513449 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a63962c7-f584-400c-894a-5c7bba374d68 req-51cfb162-36fa-4b7d-82c6-2c4d0729e0e0 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0930233 May 20 04:25:31.564656 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a63962c7-f584-400c-894a-5c7bba374d68 req-3c54575b-2fb2-47cf-ac72-8893a2c3f6fc tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0424867 May 20 04:25:31.629083 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f1b6ec79-20a9-4d16-8c89-05f98510a33b req-d47d0d47-050a-4319-94d7-a561e5350555 service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1469417 May 20 04:25:31.772747 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0a17220a-70f7-4546-8a16-90fb4e111c9b req-974860b9-2589-42d5-a45e-499e8a97f7d0 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:31.774225 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0a17220a-70f7-4546-8a16-90fb4e111c9b req-974860b9-2589-42d5-a45e-499e8a97f7d0 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=0e3baa0ea24d42f3ab59e40845b350fd&device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 1096 time: 0.0934489 May 20 04:25:31.910194 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0a17220a-70f7-4546-8a16-90fb4e111c9b req-5679e1ba-d756-4d3f-8881-dd1ccfd97a2b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:31.911607 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0a17220a-70f7-4546-8a16-90fb4e111c9b req-5679e1ba-d756-4d3f-8881-dd1ccfd97a2b service neutron] 10.222.0.74 "GET /v2.0/networks?id=14c537db-01aa-4360-a348-47f94c75e03b HTTP/1.1" status: 200 len: 911 time: 0.1288943 May 20 04:25:31.979904 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0a17220a-70f7-4546-8a16-90fb4e111c9b req-d8058468-bca7-4fa5-accf-33390389833a service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=4a6aa217-4367-4b88-9233-0a8aeee2352d HTTP/1.1" status: 200 len: 217 time: 0.0597420 May 20 04:25:32.048787 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0a17220a-70f7-4546-8a16-90fb4e111c9b req-c8ffcd29-4a03-47eb-bc96-424e37ae1a17 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:32.050269 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0a17220a-70f7-4546-8a16-90fb4e111c9b req-c8ffcd29-4a03-47eb-bc96-424e37ae1a17 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=cdea475c-2929-4ae4-9271-87cc2fb9a378 HTTP/1.1" status: 200 len: 849 time: 0.0602994 May 20 04:25:32.129998 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0a17220a-70f7-4546-8a16-90fb4e111c9b req-0894f819-f422-4ab3-adb9-91e9c9647b7b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:32.131775 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0a17220a-70f7-4546-8a16-90fb4e111c9b req-0894f819-f422-4ab3-adb9-91e9c9647b7b service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=14c537db-01aa-4360-a348-47f94c75e03b&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0743163 May 20 04:25:32.356656 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0a17220a-70f7-4546-8a16-90fb4e111c9b req-bea23557-ff6a-4d27-a840-bde348ad68c1 service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2156155 May 20 04:25:32.510553 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0a17220a-70f7-4546-8a16-90fb4e111c9b req-4b475026-b252-423f-b200-473689229a29 service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1454978 May 20 04:25:32.533775 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c05baf72-0787-4b7c-9307-a4a318e1b152 req-b34c9e6d-763b-457f-91a4-1e2e3c6c2fff tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:32.536622 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c05baf72-0787-4b7c-9307-a4a318e1b152 req-b34c9e6d-763b-457f-91a4-1e2e3c6c2fff tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 943 time: 0.0928547 May 20 04:25:32.588820 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c05baf72-0787-4b7c-9307-a4a318e1b152 req-a020985a-b542-48de-b3b9-eca604b8faf2 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0441442 May 20 04:25:33.089314 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d950d61c-0341-43eb-8a85-0a78c2f72b32 req-0db89b8d-c774-4868-858c-6bcb7f592eab tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:33.090749 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d950d61c-0341-43eb-8a85-0a78c2f72b32 req-0db89b8d-c774-4868-858c-6bcb7f592eab tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0853977 May 20 04:25:33.140683 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d950d61c-0341-43eb-8a85-0a78c2f72b32 req-6ca42421-3624-48de-a2d1-de91e735a43b tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0436251 May 20 04:25:33.788200 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0c5d96b9-1acf-4b4a-87f4-34385a6a47ae req-09f890ed-3d5a-4392-af18-742f47d3f7c4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:33.789629 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0c5d96b9-1acf-4b4a-87f4-34385a6a47ae req-09f890ed-3d5a-4392-af18-742f47d3f7c4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 943 time: 0.0850220 May 20 04:25:33.838916 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0c5d96b9-1acf-4b4a-87f4-34385a6a47ae req-7c334dcf-476d-40ac-8903-cca243ada180 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0409586 May 20 04:25:34.325539 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-65301ccf-82ba-447c-97af-187ef23226b8 req-d0dfda11-ff8a-4939-88af-184899a61fd9 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:34.327011 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-65301ccf-82ba-447c-97af-187ef23226b8 req-d0dfda11-ff8a-4939-88af-184899a61fd9 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0785377 May 20 04:25:34.387665 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-65301ccf-82ba-447c-97af-187ef23226b8 req-2b0b467b-4244-493d-8b1a-e9daedfd684f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0540912 May 20 04:25:35.029783 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-673cb615-4c5a-40fe-9a41-c19f8a71be1d req-4c54827e-b660-40be-94e3-e64984f3c989 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:35.031169 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-673cb615-4c5a-40fe-9a41-c19f8a71be1d req-4c54827e-b660-40be-94e3-e64984f3c989 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 943 time: 0.0843720 May 20 04:25:35.080593 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-673cb615-4c5a-40fe-9a41-c19f8a71be1d req-e397df8c-87cc-4d12-bd86-318e7c681afb tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0405824 May 20 04:25:35.557170 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:35.577390 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-925a2500-2936-4891-aeac-6b79136656c9 req-1eb8b941-5cc5-4ad8-b08d-ca3443eb9983 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:35.578827 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-925a2500-2936-4891-aeac-6b79136656c9 req-1eb8b941-5cc5-4ad8-b08d-ca3443eb9983 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0790770 May 20 04:25:35.630562 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-925a2500-2936-4891-aeac-6b79136656c9 req-32d16ae4-6840-44b6-baf8-a02e5a2f2655 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0457118 May 20 04:25:36.272539 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9eb91ffb-34ee-4461-a565-a1735cd2ae4d req-05fe20c4-8636-428b-b613-b943cf928672 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:36.273972 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9eb91ffb-34ee-4461-a565-a1735cd2ae4d req-05fe20c4-8636-428b-b613-b943cf928672 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 943 time: 0.0844214 May 20 04:25:36.319735 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9eb91ffb-34ee-4461-a565-a1735cd2ae4d req-10d6ce39-be35-489f-8de5-9b860081b2fc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0398371 May 20 04:25:36.896687 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b6119914-1072-4be6-8318-b67ecb4772f7 req-b2a3a8ca-35b4-4373-9d66-ca48c4db1acd tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:36.898142 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b6119914-1072-4be6-8318-b67ecb4772f7 req-b2a3a8ca-35b4-4373-9d66-ca48c4db1acd tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1342130 May 20 04:25:36.952609 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b6119914-1072-4be6-8318-b67ecb4772f7 req-1d6ef62e-0f44-46c2-a494-b165268cccf3 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0481822 May 20 04:25:36.959792 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3d87c9f1-f648-473f-b001-1fa519642ca1 req-2a71f5ab-cffb-43e2-ad3c-26234dca3212 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:36.961227 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3d87c9f1-f648-473f-b001-1fa519642ca1 req-2a71f5ab-cffb-43e2-ad3c-26234dca3212 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 945 time: 0.1339931 May 20 04:25:37.014649 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3d87c9f1-f648-473f-b001-1fa519642ca1 req-172fdac6-cef8-44e7-bfd4-b1fc6b12a0fe tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0466886 May 20 04:25:37.533276 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-801fdda6-621f-46bb-8ec8-3f4f8e6bcb70 req-7cf629a0-ba82-484a-a54b-3d4b6876f755 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:37.534682 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-801fdda6-621f-46bb-8ec8-3f4f8e6bcb70 req-7cf629a0-ba82-484a-a54b-3d4b6876f755 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 943 time: 0.0950015 May 20 04:25:37.578986 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-801fdda6-621f-46bb-8ec8-3f4f8e6bcb70 req-331e6e84-a71e-4e3c-bf57-585643632e62 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0384643 May 20 04:25:38.218893 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8161dc0c-d33f-4919-959a-a4820f0cfee8 req-5f29a23c-9049-4de7-95eb-a4b87f787f93 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:38.220341 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8161dc0c-d33f-4919-959a-a4820f0cfee8 req-5f29a23c-9049-4de7-95eb-a4b87f787f93 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1463122 May 20 04:25:38.255101 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ec456cc9-230b-43dd-8099-89ab64adbba6 req-97fb6f39-e5f0-4a17-903c-f4adaeb9e6c6 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:38.256510 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ec456cc9-230b-43dd-8099-89ab64adbba6 req-97fb6f39-e5f0-4a17-903c-f4adaeb9e6c6 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 945 time: 0.1208763 May 20 04:25:38.366010 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8161dc0c-d33f-4919-959a-a4820f0cfee8 req-20acdc0e-7210-462a-b52f-88860d8f9f58 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1382122 May 20 04:25:38.385843 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ec456cc9-230b-43dd-8099-89ab64adbba6 req-8c7a7eb6-bcdb-4e3c-8230-334c1d50249a tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1143410 May 20 04:25:38.768367 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9612e9d2-1995-483f-b6c1-5ea853f3bb6d req-8d270169-4c02-4d66-b9df-a66bf8def6f6 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:38.769813 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9612e9d2-1995-483f-b6c1-5ea853f3bb6d req-8d270169-4c02-4d66-b9df-a66bf8def6f6 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 943 time: 0.0833538 May 20 04:25:38.815369 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9612e9d2-1995-483f-b6c1-5ea853f3bb6d req-1aedbf79-5fad-4547-b491-fa63b299576b tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0393450 May 20 04:25:39.389407 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ddb1efa1-8490-4dd3-843f-cace65f447a8 req-6a75efaf-9ab3-4b24-a62b-e5362659e3ad tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:39.390204 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-ddb1efa1-8490-4dd3-843f-cace65f447a8 req-6a75efaf-9ab3-4b24-a62b-e5362659e3ad tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Synchronizing usage tracker for tenant:51fcbc78284541db832e4780e76158af on resource:port {{(pid=91644) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:25:39.402424 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-ddb1efa1-8490-4dd3-843f-cace65f447a8 req-6a75efaf-9ab3-4b24-a62b-e5362659e3ad tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Unset dirty status for tenant:51fcbc78284541db832e4780e76158af on resource:port {{(pid=91644) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:25:39.403559 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-ddb1efa1-8490-4dd3-843f-cace65f447a8 req-6a75efaf-9ab3-4b24-a62b-e5362659e3ad tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.74 "GET /v2.0/ports?device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 945 time: 0.1113510 May 20 04:25:39.539977 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [req-ddb1efa1-8490-4dd3-843f-cace65f447a8 req-c5d3b055-34ea-49b5-a864-db1e10f6eeb2 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Deleting port a859e036-1f75-40cc-b35b-68bdda10b71f {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:25:39.540333 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-ddb1efa1-8490-4dd3-843f-cace65f447a8 req-c5d3b055-34ea-49b5-a864-db1e10f6eeb2 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:39.621213 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-36c4926b-5f71-4490-a98b-1d745b06d691 req-bdd4a94e-733e-4995-a7c2-8f15c0f6278a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:39.622658 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-36c4926b-5f71-4490-a98b-1d745b06d691 req-bdd4a94e-733e-4995-a7c2-8f15c0f6278a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1475513 May 20 04:25:39.634403 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1e3c9a68-96ee-4166-8396-739f1536b7ee req-1ea37fd1-0c0d-445f-8834-9e514325d021 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:39.635840 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1e3c9a68-96ee-4166-8396-739f1536b7ee req-1ea37fd1-0c0d-445f-8834-9e514325d021 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 945 time: 0.1210134 May 20 04:25:39.675848 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-ddb1efa1-8490-4dd3-843f-cace65f447a8 req-c5d3b055-34ea-49b5-a864-db1e10f6eeb2 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] For port a859e036-1f75-40cc-b35b-68bdda10b71f, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=a859e036-1f75-40cc-b35b-68bdda10b71f,segment=NetworkSegment(6fe99325-a72c-48bf-abc7-9bbcc4b753ff),segment_id=6fe99325-a72c-48bf-abc7-9bbcc4b753ff)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:25:39.676088 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-ddb1efa1-8490-4dd3-843f-cace65f447a8 req-c5d3b055-34ea-49b5-a864-db1e10f6eeb2 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:39.704813 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-36c4926b-5f71-4490-a98b-1d745b06d691 req-568ee107-2bb4-47d5-8696-ddc1f4c86130 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0680313 May 20 04:25:39.710277 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1e3c9a68-96ee-4166-8396-739f1536b7ee req-0bd8523d-e303-412c-a90f-11409fa097f5 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0608625 May 20 04:25:39.809645 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [req-ddb1efa1-8490-4dd3-843f-cace65f447a8 req-c5d3b055-34ea-49b5-a864-db1e10f6eeb2 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Calling delete_port for a859e036-1f75-40cc-b35b-68bdda10b71f owned by compute:nova {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:25:39.990822 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-ddb1efa1-8490-4dd3-843f-cace65f447a8 req-c5d3b055-34ea-49b5-a864-db1e10f6eeb2 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:39.996784 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:40.002884 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['849868be-af51-4967-9a6c-93d23cecb450', '643e5696-1a48-46bc-9a75-8b39f176b597', 'b6bee055-19d1-4e07-bd15-16290eac96d9', 'a859e036-1f75-40cc-b35b-68bdda10b71f'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:25:40.015382 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-88f3612a-c43c-4f49-b449-14d38c69789d req-b702f0dc-9cf2-428a-a0b7-1da27f8f2864 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:40.016943 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-88f3612a-c43c-4f49-b449-14d38c69789d req-b702f0dc-9cf2-428a-a0b7-1da27f8f2864 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 943 time: 0.0826766 May 20 04:25:40.047992 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c5d3b055-34ea-49b5-a864-db1e10f6eeb2 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Pushing event deleted for resources: {'Port': ['ID=a859e036-1f75-40cc-b35b-68bdda10b71f,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:40.051158 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-c5d3b055-34ea-49b5-a864-db1e10f6eeb2 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:40.077500 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-88f3612a-c43c-4f49-b449-14d38c69789d req-cd5ab9bc-2cf9-4984-a169-ca77c9f6a80c tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0547931 May 20 04:25:40.148769 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-ddb1efa1-8490-4dd3-843f-cace65f447a8 req-c5d3b055-34ea-49b5-a864-db1e10f6eeb2 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] No DVR routers for this DVR port a859e036-1f75-40cc-b35b-68bdda10b71f on host hv-cinder-78240 {{(pid=91644) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:25:40.151856 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-ddb1efa1-8490-4dd3-843f-cace65f447a8 req-c5d3b055-34ea-49b5-a864-db1e10f6eeb2 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:40.154519 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '2f093c2c-1922-4116-95f2-a2e99555016d', 'name': 'network-vif-deleted', 'tag': 'a859e036-1f75-40cc-b35b-68bdda10b71f'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:25:40.176013 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-ddb1efa1-8490-4dd3-843f-cace65f447a8 req-c5d3b055-34ea-49b5-a864-db1e10f6eeb2 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.74 "DELETE /v2.0/ports/a859e036-1f75-40cc-b35b-68bdda10b71f HTTP/1.1" status: 204 len: 173 time: 0.7659988 May 20 04:25:40.220202 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '2f093c2c-1922-4116-95f2-a2e99555016d', 'name': 'network-vif-deleted', 'tag': 'a859e036-1f75-40cc-b35b-68bdda10b71f', 'status': 'completed', 'code': 200} May 20 04:25:40.790452 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 4a6aa217-4367-4b88-9233-0a8aeee2352d details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:25:40.926186 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5a4de411-0334-4cf3-b746-0f31bf25cedb req-bd916eaa-c97d-4c1e-ab55-f205409d32e0 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=2f093c2c-1922-4116-95f2-a2e99555016d HTTP/1.1" status: 200 len: 210 time: 0.0537677 May 20 04:25:40.935902 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 4a6aa217-4367-4b88-9233-0a8aeee2352d, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=4a6aa217-4367-4b88-9233-0a8aeee2352d,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:25:40.950748 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': '4a6aa217-4367-4b88-9233-0a8aeee2352d', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'port_id': '4a6aa217-4367-4b88-9233-0a8aeee2352d', 'mac_address': 'fa:16:3e:aa:c3:bd', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 841, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.9'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:25:40.971009 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f72b277c-cf16-4307-aa6f-b12bb85c9e96 req-a6a61288-b79f-4114-8e06-c2e8718f3ab2 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:40.972531 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f72b277c-cf16-4307-aa6f-b12bb85c9e96 req-a6a61288-b79f-4114-8e06-c2e8718f3ab2 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1321898 May 20 04:25:41.012370 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:41.019063 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f72b277c-cf16-4307-aa6f-b12bb85c9e96 req-0582d508-6868-4521-a63b-23db60d487d5 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0398881 May 20 04:25:41.030152 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 4a6aa217-4367-4b88-9233-0a8aeee2352d {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:25:41.046869 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 4a6aa217-4367-4b88-9233-0a8aeee2352d, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=4a6aa217-4367-4b88-9233-0a8aeee2352d,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:25:41.051350 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:41.053155 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:41.132052 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=4a6aa217-4367-4b88-9233-0a8aeee2352d,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:41.135447 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.082s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:41.291551 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-45df19d2-e280-4ba3-8139-b97228821d9c req-f64ac334-ec87-41a2-a449-d5a097deef15 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:41.293016 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-45df19d2-e280-4ba3-8139-b97228821d9c req-f64ac334-ec87-41a2-a449-d5a097deef15 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 944 time: 0.0990062 May 20 04:25:41.339509 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-45df19d2-e280-4ba3-8139-b97228821d9c req-472c3ea7-7423-4deb-b706-6736bb5e94e0 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0389049 May 20 04:25:41.846349 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['4a6aa217-4367-4b88-9233-0a8aeee2352d'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:25:42.218948 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0fc6b58f-ce3c-4266-8e07-7d80c2c087b3 req-7ea97cba-3f90-4f5f-b78a-0cf07448c158 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:42.220456 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0fc6b58f-ce3c-4266-8e07-7d80c2c087b3 req-7ea97cba-3f90-4f5f-b78a-0cf07448c158 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0841236 May 20 04:25:42.264082 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0fc6b58f-ce3c-4266-8e07-7d80c2c087b3 req-fc124828-7990-47ae-94eb-8da158eaad4f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0379176 May 20 04:25:42.525997 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4a2ec4f6-e47e-4f88-bfdd-4c2c760addee req-781059e2-5f07-4524-ba32-84f4bd7d6df8 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:42.527404 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4a2ec4f6-e47e-4f88-bfdd-4c2c760addee req-781059e2-5f07-4524-ba32-84f4bd7d6df8 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 944 time: 0.0819688 May 20 04:25:42.574412 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4a2ec4f6-e47e-4f88-bfdd-4c2c760addee req-f137e450-313d-4582-911b-9cb0c378f127 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0382471 May 20 04:25:42.601809 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-76031b9d-4557-4f0d-8149-a11ec4179b76 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:42.808389 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device a859e036-1f75-40cc-b35b-68bdda10b71f no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:25:42.843447 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: a859e036-1f75-40cc-b35b-68bdda10b71f {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:25:42.843796 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device a859e036-1f75-40cc-b35b-68bdda10b71f not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:25:43.459041 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a2d390c5-a034-4096-bf0b-c18a68e7a945 req-3ef2f23c-e7f6-4f79-b873-3b734d76fbae tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:43.460625 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a2d390c5-a034-4096-bf0b-c18a68e7a945 req-3ef2f23c-e7f6-4f79-b873-3b734d76fbae tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0866029 May 20 04:25:43.506528 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a2d390c5-a034-4096-bf0b-c18a68e7a945 req-8af532f9-f453-43b9-bd87-776224f2af02 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0387318 May 20 04:25:43.682609 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-17c283e7-93c6-4541-895b-a2b36d840adb req-906c67d1-fc2f-4af2-8045-5282fbed6e6c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:43.684127 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-17c283e7-93c6-4541-895b-a2b36d840adb req-906c67d1-fc2f-4af2-8045-5282fbed6e6c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 945 time: 0.0823281 May 20 04:25:43.733492 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-17c283e7-93c6-4541-895b-a2b36d840adb req-49156862-ebb7-4d09-b356-f5c2bc76af3c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0421755 May 20 04:25:43.869576 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e5682fc0-2417-4c35-aa9b-ea00ee2f3f3b req-d11cb456-6587-4749-941a-9e02de854c76 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:43.871139 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e5682fc0-2417-4c35-aa9b-ea00ee2f3f3b req-d11cb456-6587-4749-941a-9e02de854c76 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 944 time: 0.0824220 May 20 04:25:43.917766 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e5682fc0-2417-4c35-aa9b-ea00ee2f3f3b req-a6c1430c-6090-4214-a5f8-c6775b85abde tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0385106 May 20 04:25:44.704692 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3b74a9e5-3c75-4f93-9dd9-2efd32b59a38 req-8cddb738-2f9a-492e-895a-8be7bd11752a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:44.706253 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3b74a9e5-3c75-4f93-9dd9-2efd32b59a38 req-8cddb738-2f9a-492e-895a-8be7bd11752a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0857711 May 20 04:25:44.751559 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3b74a9e5-3c75-4f93-9dd9-2efd32b59a38 req-2bd6a610-0407-481f-89af-19676b58ee23 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0394135 May 20 04:25:44.800206 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:25:44.806069 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 4a6aa217-4367-4b88-9233-0a8aeee2352d up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:25:44.875913 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port 4a6aa217-4367-4b88-9233-0a8aeee2352d completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:25:44.880666 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port 4a6aa217-4367-4b88-9233-0a8aeee2352d triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:44.881039 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:44.932036 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5cc71852-b406-469f-8de5-a3ac7bc39318 req-4abca945-92ae-49f0-971a-d71e788ea3d5 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:44.933494 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5cc71852-b406-469f-8de5-a3ac7bc39318 req-4abca945-92ae-49f0-971a-d71e788ea3d5 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 945 time: 0.0831177 May 20 04:25:44.982596 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5cc71852-b406-469f-8de5-a3ac7bc39318 req-464350f2-b7af-4fe0-bf9f-3691babb23fd tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0432653 May 20 04:25:45.009867 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:45.014110 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:45.041493 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '4a6aa217-4367-4b88-9233-0a8aeee2352d'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:25:45.053232 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 4a6aa217-4367-4b88-9233-0a8aeee2352d, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=4a6aa217-4367-4b88-9233-0a8aeee2352d,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:25:45.057747 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:45.059773 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:45.097665 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '4a6aa217-4367-4b88-9233-0a8aeee2352d', 'code': 200} May 20 04:25:45.113123 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e34631f2-fe54-4c90-9b9b-f1331217d0ca req-8cc691b4-ca11-4096-b743-2a311b2d9b13 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:45.115198 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e34631f2-fe54-4c90-9b9b-f1331217d0ca req-8cc691b4-ca11-4096-b743-2a311b2d9b13 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 945 time: 0.0894344 May 20 04:25:45.133992 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-a80d30e9-a969-481c-84c9-6635c0f129b9 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:45.140236 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=4a6aa217-4367-4b88-9233-0a8aeee2352d,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:45.143297 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:45.173204 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e34631f2-fe54-4c90-9b9b-f1331217d0ca req-26ade06c-cc43-4835-ac59-a4764eef84fc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0449526 May 20 04:25:45.436628 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3c0a0207-2218-40ce-9d8a-2f30b52b7dd1 req-c94cd989-8b44-4853-8e81-d4b95eac493e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:45.437997 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-3c0a0207-2218-40ce-9d8a-2f30b52b7dd1 req-c94cd989-8b44-4853-8e81-d4b95eac493e service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 1098 time: 0.0757236 May 20 04:25:45.485538 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-3c0a0207-2218-40ce-9d8a-2f30b52b7dd1 req-4d43f87b-4773-478e-a1c8-7f342886ab17 service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0385327 May 20 04:25:45.956733 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-526dd23b-f261-40a6-8558-1e8452f37073 req-017aae6f-4270-442b-9a8c-6bbef45293a9 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:45.958176 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-526dd23b-f261-40a6-8558-1e8452f37073 req-017aae6f-4270-442b-9a8c-6bbef45293a9 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0971746 May 20 04:25:46.005537 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-526dd23b-f261-40a6-8558-1e8452f37073 req-8776c1df-b80f-426d-a4c5-d3b3ed1898a7 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0414937 May 20 04:25:46.174562 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:46.181939 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-308f670d-4e2f-41f3-a151-29fe42cc0d27 req-d40b367e-af19-447b-b9d2-313a2dc1fd97 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:46.183426 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-308f670d-4e2f-41f3-a151-29fe42cc0d27 req-d40b367e-af19-447b-b9d2-313a2dc1fd97 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 945 time: 0.0842416 May 20 04:25:46.230488 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-308f670d-4e2f-41f3-a151-29fe42cc0d27 req-7644d55f-34cd-47e2-bacf-dc93c545c9ae tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0381091 May 20 04:25:46.468906 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-67d9270b-2060-4d49-93c5-da69540af91e req-608505fc-7194-44b4-bef5-dd20f14e35e1 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:46.470328 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-67d9270b-2060-4d49-93c5-da69540af91e req-608505fc-7194-44b4-bef5-dd20f14e35e1 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 945 time: 0.1469419 May 20 04:25:46.507142 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1f99a538-a1d9-4544-895c-e4e8553f174e req-39292bd3-8838-412b-b97d-714e641efa86 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:46.507737 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-1f99a538-a1d9-4544-895c-e4e8553f174e req-39292bd3-8838-412b-b97d-714e641efa86 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Synchronizing usage tracker for tenant:51fcbc78284541db832e4780e76158af on resource:port {{(pid=91644) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:25:46.515612 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-67d9270b-2060-4d49-93c5-da69540af91e req-97a5751e-8a54-4ee8-8c5d-2b7655f28d71 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0384674 May 20 04:25:46.524404 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-1f99a538-a1d9-4544-895c-e4e8553f174e req-39292bd3-8838-412b-b97d-714e641efa86 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Unset dirty status for tenant:51fcbc78284541db832e4780e76158af on resource:port {{(pid=91644) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:25:46.525664 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-1f99a538-a1d9-4544-895c-e4e8553f174e req-39292bd3-8838-412b-b97d-714e641efa86 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.74 "GET /v2.0/ports?device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 945 time: 0.0962758 May 20 04:25:46.665827 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [req-1f99a538-a1d9-4544-895c-e4e8553f174e req-d99446f4-00e6-4dab-8f98-ef2fd73bd4b9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Deleting port b6bee055-19d1-4e07-bd15-16290eac96d9 {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:25:46.666165 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-1f99a538-a1d9-4544-895c-e4e8553f174e req-d99446f4-00e6-4dab-8f98-ef2fd73bd4b9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:46.798949 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-1f99a538-a1d9-4544-895c-e4e8553f174e req-d99446f4-00e6-4dab-8f98-ef2fd73bd4b9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] For port b6bee055-19d1-4e07-bd15-16290eac96d9, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b6bee055-19d1-4e07-bd15-16290eac96d9,segment=NetworkSegment(6fe99325-a72c-48bf-abc7-9bbcc4b753ff),segment_id=6fe99325-a72c-48bf-abc7-9bbcc4b753ff)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:25:46.799405 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-1f99a538-a1d9-4544-895c-e4e8553f174e req-d99446f4-00e6-4dab-8f98-ef2fd73bd4b9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:46.992450 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [req-1f99a538-a1d9-4544-895c-e4e8553f174e req-d99446f4-00e6-4dab-8f98-ef2fd73bd4b9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Calling delete_port for b6bee055-19d1-4e07-bd15-16290eac96d9 owned by compute:nova {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:25:47.108308 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-1f99a538-a1d9-4544-895c-e4e8553f174e req-d99446f4-00e6-4dab-8f98-ef2fd73bd4b9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:47.114366 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-5bfc6fd0-9a12-4f85-b9b3-ce12c5f663d7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:47.120429 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['849868be-af51-4967-9a6c-93d23cecb450', '643e5696-1a48-46bc-9a75-8b39f176b597'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:25:47.164667 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d99446f4-00e6-4dab-8f98-ef2fd73bd4b9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Pushing event deleted for resources: {'Port': ['ID=b6bee055-19d1-4e07-bd15-16290eac96d9,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:47.164834 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d99446f4-00e6-4dab-8f98-ef2fd73bd4b9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:47.196293 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-125310c9-a373-4be5-a31b-25e6ac4c40a9 req-7d8c3dfd-535e-497d-986d-963a6b76edf9 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:47.198252 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-125310c9-a373-4be5-a31b-25e6ac4c40a9 req-7d8c3dfd-535e-497d-986d-963a6b76edf9 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0831640 May 20 04:25:47.240952 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-1f99a538-a1d9-4544-895c-e4e8553f174e req-d99446f4-00e6-4dab-8f98-ef2fd73bd4b9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] No DVR routers for this DVR port b6bee055-19d1-4e07-bd15-16290eac96d9 on host hv-cinder-78240 {{(pid=91644) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:25:47.243465 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-1f99a538-a1d9-4544-895c-e4e8553f174e req-d99446f4-00e6-4dab-8f98-ef2fd73bd4b9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:47.246262 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'ad654613-8b28-495c-9d73-8e92df6e2464', 'name': 'network-vif-deleted', 'tag': 'b6bee055-19d1-4e07-bd15-16290eac96d9'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:25:47.247099 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-125310c9-a373-4be5-a31b-25e6ac4c40a9 req-dedcc4c8-2cb4-4ba7-911f-b8d072833029 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0425601 May 20 04:25:47.263369 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-1f99a538-a1d9-4544-895c-e4e8553f174e req-d99446f4-00e6-4dab-8f98-ef2fd73bd4b9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.74 "DELETE /v2.0/ports/b6bee055-19d1-4e07-bd15-16290eac96d9 HTTP/1.1" status: 204 len: 173 time: 0.7315791 May 20 04:25:47.411640 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ddd9b4e5-bf8f-4fa7-a80b-3bf22f557f28 req-2cd2c955-8673-4127-bad0-a2e5e931e43c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=ad654613-8b28-495c-9d73-8e92df6e2464 HTTP/1.1" status: 200 len: 210 time: 0.0516217 May 20 04:25:47.470975 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'ad654613-8b28-495c-9d73-8e92df6e2464', 'name': 'network-vif-deleted', 'tag': 'b6bee055-19d1-4e07-bd15-16290eac96d9', 'status': 'completed', 'code': 200} May 20 04:25:47.703344 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ad5d8faf-1765-4360-90be-8efda2f10f84 req-518a5001-0ce9-48dd-92bf-4562f0936ade tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:47.704729 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ad5d8faf-1765-4360-90be-8efda2f10f84 req-518a5001-0ce9-48dd-92bf-4562f0936ade tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 945 time: 0.0797181 May 20 04:25:47.750749 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ad5d8faf-1765-4360-90be-8efda2f10f84 req-81aba57b-ca14-4969-80be-cd65518e7888 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0400677 May 20 04:25:47.779724 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-716a76df-4b63-4380-9978-1ae2a4622252 req-9a3acd62-1f77-477a-b03b-48ce0e8a110d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:47.781324 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-716a76df-4b63-4380-9978-1ae2a4622252 req-9a3acd62-1f77-477a-b03b-48ce0e8a110d service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 1098 time: 0.0712085 May 20 04:25:47.822203 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:47.837741 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-716a76df-4b63-4380-9978-1ae2a4622252 req-cb2e2e8c-996c-42fb-bc25-971d75cf0b23 service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0353065 May 20 04:25:48.446931 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7cd77d8f-cf45-4d33-b299-b1691d36e5f4 req-5fa0a6c7-d3d1-4dc1-a676-c362ba6ae8e7 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:48.448495 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7cd77d8f-cf45-4d33-b299-b1691d36e5f4 req-5fa0a6c7-d3d1-4dc1-a676-c362ba6ae8e7 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0878329 May 20 04:25:48.494978 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7cd77d8f-cf45-4d33-b299-b1691d36e5f4 req-06f32abe-0ed2-4da4-acf9-d296fa014e25 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0405514 May 20 04:25:48.823776 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b6bee055-19d1-4e07-bd15-16290eac96d9 no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:25:48.861591 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: b6bee055-19d1-4e07-bd15-16290eac96d9 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:25:48.862180 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b6bee055-19d1-4e07-bd15-16290eac96d9 not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:25:48.864018 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d5b3365a-5fce-4880-8a16-b8afc158330d req-c35ad9e8-1396-4204-a850-a9db70d4edc6 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:48.865494 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d5b3365a-5fce-4880-8a16-b8afc158330d req-c35ad9e8-1396-4204-a850-a9db70d4edc6 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 945 time: 0.0902510 May 20 04:25:48.910318 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d5b3365a-5fce-4880-8a16-b8afc158330d req-01650720-d654-44ab-a652-77738467a357 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0389645 May 20 04:25:49.006144 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d9468d4a-3a67-4e00-b629-120768c5d860 req-886eb1b3-f5ed-45f2-ba13-6dec1e94c3f7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:49.007570 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d9468d4a-3a67-4e00-b629-120768c5d860 req-886eb1b3-f5ed-45f2-ba13-6dec1e94c3f7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 945 time: 0.0818563 May 20 04:25:49.055977 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d9468d4a-3a67-4e00-b629-120768c5d860 req-d7f19749-00d3-48ae-89d7-5781bc29a286 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0400085 May 20 04:25:49.245359 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0be2aa59-0344-42a2-8200-a7deb6640ab8 req-9bb1e6d5-7468-4bcd-8bd2-09ea9969ae80 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:49.246826 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0be2aa59-0344-42a2-8200-a7deb6640ab8 req-9bb1e6d5-7468-4bcd-8bd2-09ea9969ae80 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 945 time: 0.0864859 May 20 04:25:49.295201 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0be2aa59-0344-42a2-8200-a7deb6640ab8 req-506996e0-ab68-477a-912b-b22febf03772 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0399809 May 20 04:25:49.318617 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 38220) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:49.661146 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-d99b6116-8756-46f3-9981-d5a87c9f4f53 tempest-TestVolumeBootPattern-1668223768 tempest-TestVolumeBootPattern-1668223768-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:49.662524 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-d99b6116-8756-46f3-9981-d5a87c9f4f53 tempest-TestVolumeBootPattern-1668223768 tempest-TestVolumeBootPattern-1668223768-project-admin] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 1093 time: 0.3429062 May 20 04:25:49.665596 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 38230) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:49.670977 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Request body: {'floatingip': {'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'port_id': '4a6aa217-4367-4b88-9233-0a8aeee2352d', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'fixed_ip_address': '172.20.0.9'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:49.677015 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:49.680231 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-315106b3-1053-4263-a645-e31bd8e1ed84 req-4cc82697-ed4f-4f2e-91a5-3bb7e1e20bea tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:49.681056 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Usage tracker for resource:floatingip and tenant:0e3baa0ea24d42f3ab59e40845b350fd is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:25:49.681678 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-315106b3-1053-4263-a645-e31bd8e1ed84 req-4cc82697-ed4f-4f2e-91a5-3bb7e1e20bea tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0791898 May 20 04:25:49.684060 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:25:49.690051 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:49.701726 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for floatingip, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:49.729166 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-315106b3-1053-4263-a645-e31bd8e1ed84 req-21cc646d-b419-4727-9ff8-b3a4ad86fc61 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0413320 May 20 04:25:49.839943 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:49.985704 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Allocated IP 172.27.0.245 (712ce8c4-8feb-4719-bf13-e06992125cd9/e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5/a2787f07-3708-4444-9177-fd660f18fb66) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:25:50.114586 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bc599b82-3c3c-4ff6-852e-d9143139a192 req-4942f04a-2e9a-42bb-b1bb-ced9f2352dcd tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:50.116783 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bc599b82-3c3c-4ff6-852e-d9143139a192 req-4942f04a-2e9a-42bb-b1bb-ced9f2352dcd tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 945 time: 0.0916836 May 20 04:25:50.140919 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:50.175411 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:50.181266 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:50.189179 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bc599b82-3c3c-4ff6-852e-d9143139a192 req-2bdf95cf-0cf5-4484-875b-a03131fd0562 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0522673 May 20 04:25:50.293143 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:25:50.297212 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:25:50.301735 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:25:50.308213 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c1117f0e-a5b4-4a7a-8492-0f8d7c348d46 req-f909fee0-7453-44e2-9fde-0807db61321a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:50.309655 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c1117f0e-a5b4-4a7a-8492-0f8d7c348d46 req-f909fee0-7453-44e2-9fde-0807db61321a tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0851333 May 20 04:25:50.341451 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event updated for resources: {'Port': ['ID=a2787f07-3708-4444-9177-fd660f18fb66,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:50.344585 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.163s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:50.355700 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c1117f0e-a5b4-4a7a-8492-0f8d7c348d46 req-b40469b4-eb4d-48b3-bfd9-7484a074e0f7 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0401168 May 20 04:25:50.429642 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2c59c9e4-7e5b-4b5a-8bb2-81fd656abf97 req-bebcb339-fac6-4f68-bf72-bfaea05d0525 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:50.431018 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-2c59c9e4-7e5b-4b5a-8bb2-81fd656abf97 req-bebcb339-fac6-4f68-bf72-bfaea05d0525 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=bec56d8703cf47f3b7193b7d1f55396d&device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 1098 time: 0.2516441 May 20 04:25:50.734955 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2c59c9e4-7e5b-4b5a-8bb2-81fd656abf97 req-faf1bde7-4e8c-4eb7-acfc-82e5edd85173 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:50.736315 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-2c59c9e4-7e5b-4b5a-8bb2-81fd656abf97 req-faf1bde7-4e8c-4eb7-acfc-82e5edd85173 service neutron] 10.222.0.74 "GET /v2.0/networks?id=36b37213-8439-447d-9eba-a78baf7c698e HTTP/1.1" status: 200 len: 920 time: 0.2730033 May 20 04:25:50.899403 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-2c59c9e4-7e5b-4b5a-8bb2-81fd656abf97 req-29a6b92d-a0e3-4b23-bd5c-74ed49363087 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=706277c6-cd8c-4146-ab91-ed9504446690 HTTP/1.1" status: 200 len: 217 time: 0.1483274 May 20 04:25:50.983655 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2c59c9e4-7e5b-4b5a-8bb2-81fd656abf97 req-46536c1e-5807-4886-bfc0-acf1cf2b4ba0 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:50.985178 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-2c59c9e4-7e5b-4b5a-8bb2-81fd656abf97 req-46536c1e-5807-4886-bfc0-acf1cf2b4ba0 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=b25ae9c8-e11d-45ee-bde2-332cc1538602 HTTP/1.1" status: 200 len: 859 time: 0.0741999 May 20 04:25:51.038214 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-2c59c9e4-7e5b-4b5a-8bb2-81fd656abf97 req-573304a7-54de-4922-b561-606f9a407a0d service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=36b37213-8439-447d-9eba-a78baf7c698e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0427756 May 20 04:25:51.194052 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for floatingip, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:51.235556 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-2c59c9e4-7e5b-4b5a-8bb2-81fd656abf97 req-6297891f-aa6d-41b1-89b5-765e8c1a9003 service neutron] 10.222.0.74 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1562994 May 20 04:25:51.372680 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:51.442919 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5b742b01-2f91-4223-b166-5e82d21e9d89 req-9413f1e9-2f61-4e06-baeb-402eb8967387 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:51.444388 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5b742b01-2f91-4223-b166-5e82d21e9d89 req-9413f1e9-2f61-4e06-baeb-402eb8967387 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 945 time: 0.1458831 May 20 04:25:51.473690 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-42f47745-a60d-478a-83fb-b0358540871d req-4ecdefba-2e87-494a-a3cc-8e6ca92f2e51 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:51.475080 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-42f47745-a60d-478a-83fb-b0358540871d req-4ecdefba-2e87-494a-a3cc-8e6ca92f2e51 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.74 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 945 time: 0.1212838 May 20 04:25:51.482264 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-2c59c9e4-7e5b-4b5a-8bb2-81fd656abf97 req-93aa3948-3ec2-4e89-ac02-06bb3f1debce service neutron] 10.222.0.74 "GET /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2142446 May 20 04:25:51.605960 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5b742b01-2f91-4223-b166-5e82d21e9d89 req-c7c73b40-3019-4a91-8c09-7eadface6521 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1529343 May 20 04:25:51.615611 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:51.621215 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:51.625147 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:51.683413 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b300f7e9-1a02-4171-8865-8210aad22b95 req-f44a846c-27da-4ea5-99d8-c1a55be7ddc6 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:51.684866 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b300f7e9-1a02-4171-8865-8210aad22b95 req-f44a846c-27da-4ea5-99d8-c1a55be7ddc6 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.2088985 May 20 04:25:51.708684 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-42f47745-a60d-478a-83fb-b0358540871d req-1cedfb6e-1f64-4405-8587-880e3b5e29f9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Deleting port 643e5696-1a48-46bc-9a75-8b39f176b597 {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:25:51.709020 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-42f47745-a60d-478a-83fb-b0358540871d req-1cedfb6e-1f64-4405-8587-880e3b5e29f9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:51.730483 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event updated for resources: {'Port': ['ID=a2787f07-3708-4444-9177-fd660f18fb66,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:51.733213 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.108s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:51.742408 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-8003251'] for floatingip, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:51.822493 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b300f7e9-1a02-4171-8865-8210aad22b95 req-b764f1e9-5a4f-4909-b4c4-cb446f365b27 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1277368 May 20 04:25:51.864900 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-42f47745-a60d-478a-83fb-b0358540871d req-1cedfb6e-1f64-4405-8587-880e3b5e29f9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] For port 643e5696-1a48-46bc-9a75-8b39f176b597, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=643e5696-1a48-46bc-9a75-8b39f176b597,segment=NetworkSegment(6fe99325-a72c-48bf-abc7-9bbcc4b753ff),segment_id=6fe99325-a72c-48bf-abc7-9bbcc4b753ff)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:25:51.865139 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-42f47745-a60d-478a-83fb-b0358540871d req-1cedfb6e-1f64-4405-8587-880e3b5e29f9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:51.922774 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.db.l3_db [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Floating IP d2b40541-a315-4701-99bd-be17179c9b8b associated. External IP: 172.27.0.245, port: 4a6aa217-4367-4b88-9233-0a8aeee2352d. May 20 04:25:52.060971 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-42f47745-a60d-478a-83fb-b0358540871d req-1cedfb6e-1f64-4405-8587-880e3b5e29f9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Calling delete_port for 643e5696-1a48-46bc-9a75-8b39f176b597 owned by compute:nova {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:25:52.174729 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-42f47745-a60d-478a-83fb-b0358540871d req-1cedfb6e-1f64-4405-8587-880e3b5e29f9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:52.180515 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:52.191948 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['849868be-af51-4967-9a6c-93d23cecb450', '643e5696-1a48-46bc-9a75-8b39f176b597'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:25:52.226143 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1cedfb6e-1f64-4405-8587-880e3b5e29f9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Pushing event deleted for resources: {'Port': ['ID=643e5696-1a48-46bc-9a75-8b39f176b597,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:52.255817 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-1cedfb6e-1f64-4405-8587-880e3b5e29f9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.075s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:52.304325 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [req-42f47745-a60d-478a-83fb-b0358540871d req-1cedfb6e-1f64-4405-8587-880e3b5e29f9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] No DVR routers for this DVR port 643e5696-1a48-46bc-9a75-8b39f176b597 on host hv-cinder-78240 {{(pid=91643) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:25:52.308133 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-42f47745-a60d-478a-83fb-b0358540871d req-1cedfb6e-1f64-4405-8587-880e3b5e29f9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:52.311241 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'bc7b7a9f-a875-49ec-a1ed-47a7798818e7', 'name': 'network-vif-deleted', 'tag': '643e5696-1a48-46bc-9a75-8b39f176b597'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:25:52.323400 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-42f47745-a60d-478a-83fb-b0358540871d req-1cedfb6e-1f64-4405-8587-880e3b5e29f9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.74 "DELETE /v2.0/ports/643e5696-1a48-46bc-9a75-8b39f176b597 HTTP/1.1" status: 204 len: 173 time: 0.8157384 May 20 04:25:52.392356 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'bc7b7a9f-a875-49ec-a1ed-47a7798818e7', 'name': 'network-vif-deleted', 'tag': '643e5696-1a48-46bc-9a75-8b39f176b597', 'status': 'completed', 'code': 200} May 20 04:25:52.440852 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Router 4969b258-f606-4def-b666-09764c475749 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91644) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:25:52.631177 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:25:52.641646 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0befd48a-67a9-46d6-a447-478f99d91cac None None] Sync routers for ids ['4969b258-f606-4def-b666-09764c475749'] in dv-cinder-782405-9-iscsi {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:25:52.653197 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0befd48a-67a9-46d6-a447-478f99d91cac None None] Routers {'4969b258-f606-4def-b666-09764c475749'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:25:52.654006 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-0befd48a-67a9-46d6-a447-478f99d91cac None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 8, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 8, "log_agent_heartbeats": false, "routers": 8},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:25:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['4969b258-f606-4def-b666-09764c475749'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:52.659435 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:52.715387 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 996 time: 3.0484838 May 20 04:25:52.716272 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71', 'tag': '4a6aa217-4367-4b88-9233-0a8aeee2352d'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:25:52.770989 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71', 'tag': '4a6aa217-4367-4b88-9233-0a8aeee2352d', 'status': 'completed', 'code': 200} May 20 04:25:52.787454 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-e32be020-3210-4fa2-b0f3-6496cf367658 req-97c72b38-9519-494e-a53d-59475ad060f7 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Synchronizing usage tracker for tenant:51fcbc78284541db832e4780e76158af on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:25:52.838249 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-0befd48a-67a9-46d6-a447-478f99d91cac None None] Set router 4969b258-f606-4def-b666-09764c475749 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:25:52.876444 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-e32be020-3210-4fa2-b0f3-6496cf367658 req-97c72b38-9519-494e-a53d-59475ad060f7 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Unset dirty status for tenant:51fcbc78284541db832e4780e76158af on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:25:52.886637 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e32be020-3210-4fa2-b0f3-6496cf367658 req-97c72b38-9519-494e-a53d-59475ad060f7 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=bc7b7a9f-a875-49ec-a1ed-47a7798818e7 HTTP/1.1" status: 200 len: 210 time: 0.1414940 May 20 04:25:52.919470 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-82c1fb56-e273-44ec-b659-01a01c9b6086 req-2f11e9aa-81fe-4c7a-84dc-120ac643bf68 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:52.921265 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-82c1fb56-e273-44ec-b659-01a01c9b6086 req-2f11e9aa-81fe-4c7a-84dc-120ac643bf68 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=0e3baa0ea24d42f3ab59e40845b350fd&device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 1098 time: 0.1271100 May 20 04:25:53.126589 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-82c1fb56-e273-44ec-b659-01a01c9b6086 req-6e52b45a-306f-48f1-b157-e8558384bdbc service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:53.128298 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-82c1fb56-e273-44ec-b659-01a01c9b6086 req-6e52b45a-306f-48f1-b157-e8558384bdbc service neutron] 10.222.0.74 "GET /v2.0/networks?id=14c537db-01aa-4360-a348-47f94c75e03b HTTP/1.1" status: 200 len: 911 time: 0.1996613 May 20 04:25:53.142438 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-71ee87e3-7c04-4449-b49b-a1c327752461 req-555c3be9-c83e-4302-8bce-b2136ed6cc46 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:53.143957 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-71ee87e3-7c04-4449-b49b-a1c327752461 req-555c3be9-c83e-4302-8bce-b2136ed6cc46 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1832502 May 20 04:25:53.196845 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-0befd48a-67a9-46d6-a447-478f99d91cac None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'd2b40541-a315-4701-99bd-be17179c9b8b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'floating_ip_address': '172.27.0.245', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '4969b258-f606-4def-b666-09764c475749', 'port_id': '4a6aa217-4367-4b88-9233-0a8aeee2352d', 'fixed_ip_address': '172.20.0.9', 'status': 'DOWN', 'standard_attr_id': 379, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'mac_address': 'fa:16:3e:aa:c3:bd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:25:50Z', 'updated_at': '2021-05-20T04:25:50Z', 'revision_number': 0, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:50Z', 'revision_number': 5, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:53.200889 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0befd48a-67a9-46d6-a447-478f99d91cac None None] Checking router: 4969b258-f606-4def-b666-09764c475749 for host: dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:25:53.209520 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0befd48a-67a9-46d6-a447-478f99d91cac None None] Routers {'4969b258-f606-4def-b666-09764c475749'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:25:53.210231 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-0befd48a-67a9-46d6-a447-478f99d91cac None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 8, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 8, "log_agent_heartbeats": false, "routers": 8},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:25:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['4969b258-f606-4def-b666-09764c475749'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:53.274028 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-71ee87e3-7c04-4449-b49b-a1c327752461 req-1c755af5-acb3-490b-9a69-2a69c55034a4 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0745885 May 20 04:25:53.275786 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-82c1fb56-e273-44ec-b659-01a01c9b6086 req-6e27d3ff-ef17-4785-855e-dc4f94acd6e7 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=4a6aa217-4367-4b88-9233-0a8aeee2352d HTTP/1.1" status: 200 len: 999 time: 0.1309695 May 20 04:25:53.338959 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-82c1fb56-e273-44ec-b659-01a01c9b6086 req-01c91502-ccf9-4be2-9956-31e2ffb090e0 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:53.340282 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-82c1fb56-e273-44ec-b659-01a01c9b6086 req-01c91502-ccf9-4be2-9956-31e2ffb090e0 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=cdea475c-2929-4ae4-9271-87cc2fb9a378 HTTP/1.1" status: 200 len: 849 time: 0.0573647 May 20 04:25:53.392980 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-0befd48a-67a9-46d6-a447-478f99d91cac None None] Set router 4969b258-f606-4def-b666-09764c475749 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:25:53.416999 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-82c1fb56-e273-44ec-b659-01a01c9b6086 req-30151a75-07a1-4acf-9753-61d998c40ab9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:53.418475 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-82c1fb56-e273-44ec-b659-01a01c9b6086 req-30151a75-07a1-4acf-9753-61d998c40ab9 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=14c537db-01aa-4360-a348-47f94c75e03b&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0707147 May 20 04:25:53.561002 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-82c1fb56-e273-44ec-b659-01a01c9b6086 req-61b552ef-7b73-4441-911a-e7678e7b2932 service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1343243 May 20 04:25:53.697056 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-82c1fb56-e273-44ec-b659-01a01c9b6086 req-b9662662-78a0-470b-b750-87ab777230df service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1257277 May 20 04:25:53.737328 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-0befd48a-67a9-46d6-a447-478f99d91cac None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'd2b40541-a315-4701-99bd-be17179c9b8b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'floating_ip_address': '172.27.0.245', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '4969b258-f606-4def-b666-09764c475749', 'port_id': '4a6aa217-4367-4b88-9233-0a8aeee2352d', 'fixed_ip_address': '172.20.0.9', 'status': 'DOWN', 'standard_attr_id': 379, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'mac_address': 'fa:16:3e:aa:c3:bd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:25:50Z', 'updated_at': '2021-05-20T04:25:50Z', 'revision_number': 0, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:50Z', 'revision_number': 5, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:25:53.741159 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0befd48a-67a9-46d6-a447-478f99d91cac None None] The sync data for ids ['4969b258-f606-4def-b666-09764c475749'] in dv-cinder-782405-9-iscsi is: [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'd2b40541-a315-4701-99bd-be17179c9b8b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'floating_ip_address': '172.27.0.245', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '4969b258-f606-4def-b666-09764c475749', 'port_id': '4a6aa217-4367-4b88-9233-0a8aeee2352d', 'fixed_ip_address': '172.20.0.9', 'status': 'DOWN', 'standard_attr_id': 379, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'mac_address': 'fa:16:3e:aa:c3:bd', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:25:50Z', 'updated_at': '2021-05-20T04:25:50Z', 'revision_number': 0, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:50Z', 'revision_number': 5, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:25:53.916035 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-df626bb4-8bd3-4cfd-8f98-6fc9fd330025 None None] New status for floating IP d2b40541-a315-4701-99bd-be17179c9b8b: ACTIVE {{(pid=91645) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 20 04:25:54.482632 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e2e884da-f448-4c52-bc46-4233d616f3a4 req-d60e72ae-77a5-4c3e-b904-4926e00550dc tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:54.484017 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e2e884da-f448-4c52-bc46-4233d616f3a4 req-d60e72ae-77a5-4c3e-b904-4926e00550dc tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0926199 May 20 04:25:54.530688 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e2e884da-f448-4c52-bc46-4233d616f3a4 req-65742428-556f-4396-bb7e-c1791bddf127 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0405192 May 20 04:25:54.862373 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-749ab75b-0c85-4417-bbd3-d2ae7e2d5906 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-434830226&fields=id&tenant_id=39febf983bf24778a300f287dbeca829 HTTP/1.1" status: 200 len: 267 time: 0.0455897 May 20 04:25:54.885421 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 643e5696-1a48-46bc-9a75-8b39f176b597 no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:25:54.918338 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: 643e5696-1a48-46bc-9a75-8b39f176b597 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:25:54.919054 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 643e5696-1a48-46bc-9a75-8b39f176b597 not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:25:55.020619 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-2fb882c7-f15e-4c84-94ee-25b034cd11fa tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:55.021973 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-2fb882c7-f15e-4c84-94ee-25b034cd11fa tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/networks?id=e57f6e29-4637-455e-af9d-0975c45c995c HTTP/1.1" status: 200 len: 808 time: 0.1531246 May 20 04:25:55.037092 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-132485f7-5cef-4ef7-a429-b7311fa48c8b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/quotas/39febf983bf24778a300f287dbeca829 HTTP/1.1" status: 200 len: 375 time: 0.0090618 May 20 04:25:55.124191 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-557d6316-3dfe-4997-ab34-44523a908c87 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=39febf983bf24778a300f287dbeca829&fields=id HTTP/1.1" status: 200 len: 302 time: 0.0821807 May 20 04:25:55.342393 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-8c2e20f8-e9bd-4c63-aa22-f6307e857c46 service neutron] 10.222.0.39 "GET /v2.0/networks/e57f6e29-4637-455e-af9d-0975c45c995c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2049124 May 20 04:25:55.479510 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-92c0ff11-853e-4554-a038-d6e1d0b84154 service neutron] 10.222.0.39 "GET /v2.0/networks/e57f6e29-4637-455e-af9d-0975c45c995c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1301498 May 20 04:25:55.704072 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9159dc1e-6196-4cb7-a9f6-6e92bdfd88a4 req-c241a85f-e143-4e6a-9cca-6349cac4f247 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:55.705576 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9159dc1e-6196-4cb7-a9f6-6e92bdfd88a4 req-c241a85f-e143-4e6a-9cca-6349cac4f247 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 946 time: 0.0802472 May 20 04:25:55.752353 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9159dc1e-6196-4cb7-a9f6-6e92bdfd88a4 req-35dee276-e510-498b-b763-e7cb11bfde5d tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0409894 May 20 04:25:55.937189 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ec4502d-b0b4-4cb1-84cb-1629f3fb61f9 req-a265568f-9117-4f86-8912-fffaa9ea7921 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:55.938701 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0ec4502d-b0b4-4cb1-84cb-1629f3fb61f9 req-a265568f-9117-4f86-8912-fffaa9ea7921 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0876753 May 20 04:25:56.017027 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cf227c90-8428-48df-aee3-81c293ce8b10 req-589aca0e-dc25-41c0-9fb7-180925d9fcec tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 210 time: 0.0520091 May 20 04:25:56.027908 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0ec4502d-b0b4-4cb1-84cb-1629f3fb61f9 req-0a7454b5-fc98-45c4-8d80-86baab1ad346 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0829625 May 20 04:25:57.049275 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6ae895cf-3c30-4d80-a571-2d7f616d59fb req-806d3462-c3c9-4c27-b6e4-4e785c84ec8a tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:57.050729 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6ae895cf-3c30-4d80-a571-2d7f616d59fb req-806d3462-c3c9-4c27-b6e4-4e785c84ec8a tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 946 time: 0.1889174 May 20 04:25:57.099098 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-4692d5af-7c84-465a-8abf-6f84645ac621 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:57.100578 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-4692d5af-7c84-465a-8abf-6f84645ac621 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.74 "GET /v2.0/networks?id=e57f6e29-4637-455e-af9d-0975c45c995c HTTP/1.1" status: 200 len: 808 time: 0.1818871 May 20 04:25:57.120449 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6ae895cf-3c30-4d80-a571-2d7f616d59fb req-d8b0b2ba-7b66-4774-a54b-84015b1381c7 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0630872 May 20 04:25:57.176692 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-819bce9f-9129-4770-bdc2-ae8cab0472b5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.74 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=39febf983bf24778a300f287dbeca829 HTTP/1.1" status: 200 len: 379 time: 0.0538642 May 20 04:25:57.185839 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Request body: {'port': {'device_id': '30d5e70e-9acd-400c-83be-01301f56a5ff', 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c', 'admin_state_up': True, 'tenant_id': '39febf983bf24778a300f287dbeca829', 'security_groups': ['9eacb756-312a-4612-a34b-e9da9dda4f71']}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:57.258394 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-48f8bcea-2126-4d39-b3a7-5ef0604a021b req-852e5567-434c-4505-89fb-30aee562731d tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 210 time: 0.0589139 May 20 04:25:57.363080 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:25:57.367454 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Usage tracker for resource:port and tenant:39febf983bf24778a300f287dbeca829 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:25:57.370422 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Quota usage for port was recalculated. Used quota:2. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:25:57.377207 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:25:57.388538 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Made reservation on behalf of 39febf983bf24778a300f287dbeca829 for: {'port': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:25:57.392228 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:57.563772 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Allocated IP 172.20.0.5 (e57f6e29-4637-455e-af9d-0975c45c995c/49523a89-260b-4732-abff-507d74fed4c4/b858096a-44b8-4537-a167-0e9c03347445) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:25:57.775088 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:57.791816 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.provisioning_blocks [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Transition to ACTIVE for port object b858096a-44b8-4537-a167-0e9c03347445 will not be triggered until provisioned by entity DHCP. {{(pid=91643) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:25:57.816232 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:57.821946 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-1cedfb6e-1f64-4405-8587-880e3b5e29f9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:57.897055 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Network e57f6e29-4637-455e-af9d-0975c45c995c is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:25:57.898353 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:25:57.900864 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:25:57.922955 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cee84aa7-a61b-4420-be11-5e5cd6299074 req-e7de23fb-6b59-40e1-a93a-68208cb90e1c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:57.923682 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-cee84aa7-a61b-4420-be11-5e5cd6299074 req-e7de23fb-6b59-40e1-a93a-68208cb90e1c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Synchronizing usage tracker for tenant:51fcbc78284541db832e4780e76158af on resource:port {{(pid=91644) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:25:57.936026 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Pushing event updated for resources: {'Port': ['ID=b858096a-44b8-4537-a167-0e9c03347445,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:57.936673 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-cee84aa7-a61b-4420-be11-5e5cd6299074 req-e7de23fb-6b59-40e1-a93a-68208cb90e1c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Unset dirty status for tenant:51fcbc78284541db832e4780e76158af on resource:port {{(pid=91644) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:25:57.937826 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-cee84aa7-a61b-4420-be11-5e5cd6299074 req-e7de23fb-6b59-40e1-a93a-68208cb90e1c tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.74 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 946 time: 0.1136940 May 20 04:25:57.955430 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:57.955819 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:57.956170 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:57.956530 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:57.956878 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:57.957227 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:57.957571 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:57.957914 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:57.958258 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:57.958598 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:57.958963 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:57.959309 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:57.963033 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.141s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:57.985864 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:57.986539 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:58.013243 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 0.8310425 May 20 04:25:58.026696 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Request body: {'port': {'device_id': '30d5e70e-9acd-400c-83be-01301f56a5ff', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:25:58.093421 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-aba951b7-3e8b-4cfb-868d-8aea1a986c7c None None] Provisioning for port b858096a-44b8-4537-a167-0e9c03347445 completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:25:58.094892 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [req-cee84aa7-a61b-4420-be11-5e5cd6299074 req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Deleting port 849868be-af51-4967-9a6c-93d23cecb450 {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:25:58.095271 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-cee84aa7-a61b-4420-be11-5e5cd6299074 req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:58.098483 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-aba951b7-3e8b-4cfb-868d-8aea1a986c7c None None] Provisioning complete for port b858096a-44b8-4537-a167-0e9c03347445 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:25:58.099239 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-aba951b7-3e8b-4cfb-868d-8aea1a986c7c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:58.165010 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-aba951b7-3e8b-4cfb-868d-8aea1a986c7c None None] Port b858096a-44b8-4537-a167-0e9c03347445 cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:25:58.166045 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:58.239523 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-cee84aa7-a61b-4420-be11-5e5cd6299074 req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] For port 849868be-af51-4967-9a6c-93d23cecb450, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=849868be-af51-4967-9a6c-93d23cecb450,segment=NetworkSegment(6fe99325-a72c-48bf-abc7-9bbcc4b753ff),segment_id=6fe99325-a72c-48bf-abc7-9bbcc4b753ff)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:25:58.240252 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-cee84aa7-a61b-4420-be11-5e5cd6299074 req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:58.390638 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [req-cee84aa7-a61b-4420-be11-5e5cd6299074 req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Calling delete_port for 849868be-af51-4967-9a6c-93d23cecb450 owned by compute:nova {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:25:58.511801 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-cee84aa7-a61b-4420-be11-5e5cd6299074 req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:58.517368 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-8df38853-10fc-47c3-ac54-12309d85af2c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:58.524800 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['849868be-af51-4967-9a6c-93d23cecb450'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:25:58.567441 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Pushing event deleted for resources: {'Port': ['ID=849868be-af51-4967-9a6c-93d23cecb450,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:58.607572 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:58.607855 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:58.608215 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:58.608571 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:58.608937 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:58.609315 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:58.609689 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:58.610055 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:58.610423 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:58.610786 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:58.611152 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:58.611584 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:25:58.618709 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.101s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:58.653999 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-cee84aa7-a61b-4420-be11-5e5cd6299074 req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] No DVR routers for this DVR port 849868be-af51-4967-9a6c-93d23cecb450 on host hv-cinder-78240 {{(pid=91644) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:25:58.656652 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-cee84aa7-a61b-4420-be11-5e5cd6299074 req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:58.659405 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'e94a8b2b-8a29-45d4-9edb-63137be69ce4', 'name': 'network-vif-deleted', 'tag': '849868be-af51-4967-9a6c-93d23cecb450'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:25:58.673014 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c4953805-76a4-487a-bcd3-2aa536b08c1b req-1ae1b1a1-b9da-4fec-9a8d-ccea8d79eb3a tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:58.674712 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c4953805-76a4-487a-bcd3-2aa536b08c1b req-1ae1b1a1-b9da-4fec-9a8d-ccea8d79eb3a tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/ports?device_id=e94a8b2b-8a29-45d4-9edb-63137be69ce4 HTTP/1.1" status: 200 len: 946 time: 0.4293895 May 20 04:25:58.676261 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-cee84aa7-a61b-4420-be11-5e5cd6299074 req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.74 "DELETE /v2.0/ports/849868be-af51-4967-9a6c-93d23cecb450 HTTP/1.1" status: 204 len: 173 time: 0.7297955 May 20 04:25:58.728201 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5a8d87ed-76a9-4b89-8b28-6317967e698f req-da8e0748-8d32-429b-a294-2bb012c2b677 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:58.728895 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-5a8d87ed-76a9-4b89-8b28-6317967e698f req-da8e0748-8d32-429b-a294-2bb012c2b677 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Synchronizing usage tracker for tenant:39febf983bf24778a300f287dbeca829 on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:25:58.771050 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c4953805-76a4-487a-bcd3-2aa536b08c1b req-51a46a5c-a80f-48dd-8d55-9ced0d05990f tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] 10.222.0.39 "GET /v2.0/security-groups?id=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0892487 May 20 04:25:58.780812 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-5a8d87ed-76a9-4b89-8b28-6317967e698f req-da8e0748-8d32-429b-a294-2bb012c2b677 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Unset dirty status for tenant:39febf983bf24778a300f287dbeca829 on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:25:58.781909 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5a8d87ed-76a9-4b89-8b28-6317967e698f req-da8e0748-8d32-429b-a294-2bb012c2b677 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 931 time: 0.3396406 May 20 04:25:58.837730 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'e94a8b2b-8a29-45d4-9edb-63137be69ce4', 'name': 'network-vif-deleted', 'tag': '849868be-af51-4967-9a6c-93d23cecb450', 'status': 'completed', 'code': 200} May 20 04:25:58.873242 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id b858096a-44b8-4537-a167-0e9c03347445 {{(pid=91643) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:25:58.884390 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Notify callbacks [] for port, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:58.889021 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5a8d87ed-76a9-4b89-8b28-6317967e698f req-6b532893-22f1-4a52-a44c-270d7e1f9e32 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0715554 May 20 04:25:58.904255 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.provisioning_blocks [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Transition to ACTIVE for port object b858096a-44b8-4537-a167-0e9c03347445 will not be triggered until provisioned by entity L2. {{(pid=91643) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:25:58.909394 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:58.913708 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:59.069981 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Pushing event updated for resources: {'Port': ['ID=b858096a-44b8-4537-a167-0e9c03347445,revision_number=2']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:59.088109 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.174s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:59.158716 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Attempting to bind port b858096a-44b8-4537-a167-0e9c03347445 on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:25:59.159171 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Attempting to bind port b858096a-44b8-4537-a167-0e9c03347445 by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': 'c7d77442-5ac6-48cf-ab59-3f5962fc1e3e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 806, 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c'}] {{(pid=91643) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:25:59.159561 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Attempting to bind port b858096a-44b8-4537-a167-0e9c03347445 on network e57f6e29-4637-455e-af9d-0975c45c995c {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:25:59.165010 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Port b858096a-44b8-4537-a167-0e9c03347445 on network e57f6e29-4637-455e-af9d-0975c45c995c not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:25:59.165339 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Attempting to bind port b858096a-44b8-4537-a167-0e9c03347445 on network e57f6e29-4637-455e-af9d-0975c45c995c {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:25:59.171276 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 25, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:25:59.171981 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Checking segment: {'id': 'c7d77442-5ac6-48cf-ab59-3f5962fc1e3e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 806, 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91643) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:25:59.172167 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Bound using segment: {'id': 'c7d77442-5ac6-48cf-ab59-3f5962fc1e3e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 806, 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c'} {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:25:59.172698 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Bound port: b858096a-44b8-4537-a167-0e9c03347445, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'c7d77442-5ac6-48cf-ab59-3f5962fc1e3e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 806, 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c'}}] {{(pid=91643) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:25:59.173048 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:59.277284 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] For port b858096a-44b8-4537-a167-0e9c03347445, host hv-cinder-78240, cleared binding levels {{(pid=91643) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:25:59.281395 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] For port b858096a-44b8-4537-a167-0e9c03347445, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b858096a-44b8-4537-a167-0e9c03347445,segment=NetworkSegment(c7d77442-5ac6-48cf-ab59-3f5962fc1e3e),segment_id=c7d77442-5ac6-48cf-ab59-3f5962fc1e3e)] {{(pid=91643) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:25:59.286637 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:59.292842 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:25:59.346028 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:59.346666 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:59.348516 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] 10.222.0.74 "PUT /v2.0/ports/b858096a-44b8-4537-a167-0e9c03347445 HTTP/1.1" status: 200 len: 1093 time: 1.3268647 May 20 04:25:59.356273 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '30d5e70e-9acd-400c-83be-01301f56a5ff', 'tag': 'b858096a-44b8-4537-a167-0e9c03347445'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:25:59.386246 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Pushing event updated for resources: {'Port': ['ID=b858096a-44b8-4537-a167-0e9c03347445,revision_number=3']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:25:59.391370 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.099s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:25:59.448513 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '30d5e70e-9acd-400c-83be-01301f56a5ff', 'tag': 'b858096a-44b8-4537-a167-0e9c03347445', 'status': 'completed', 'code': 200} May 20 04:25:59.500966 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-b242dd47-2c29-41d2-b6aa-9e0fb26b8790 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:59.502522 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-b242dd47-2c29-41d2-b6aa-9e0fb26b8790 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=39febf983bf24778a300f287dbeca829&device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 1096 time: 0.0877695 May 20 04:25:59.571158 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-bfb4b1d1-7d9d-4fb2-8840-39338f4bceb8 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=b858096a-44b8-4537-a167-0e9c03347445 HTTP/1.1" status: 200 len: 217 time: 0.0604198 May 20 04:25:59.638190 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-281e9394-6ad9-4888-ae74-cc5ccdbe3133 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:59.639477 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-281e9394-6ad9-4888-ae74-cc5ccdbe3133 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=49523a89-260b-4732-abff-507d74fed4c4 HTTP/1.1" status: 200 len: 847 time: 0.0615222 May 20 04:25:59.717099 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-d28ee39f-1221-494a-9b06-b76bafc330b2 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:25:59.718530 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-d28ee39f-1221-494a-9b06-b76bafc330b2 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=e57f6e29-4637-455e-af9d-0975c45c995c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0721588 May 20 04:25:59.851244 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-6c47823a-7a54-41b9-8e1b-7ca265388503 service neutron] 10.222.0.74 "GET /v2.0/networks/e57f6e29-4637-455e-af9d-0975c45c995c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1233473 May 20 04:25:59.871449 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 38376) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:25:59.933854 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_v2 [None req-6350424d-0718-4e5b-948b-ded8293f840f admin admin] Deleting subnet 0f918085-8d33-4e36-a59e-7f270a2b1874 {{(pid=91644) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 20 04:25:59.936992 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6350424d-0718-4e5b-948b-ded8293f840f admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:25:59.991145 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6350424d-0718-4e5b-948b-ded8293f840f admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-853257'] for subnet, precommit_delete_associations {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:00.050493 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4ce87987-193a-4ca5-a725-f95fc5dccc8d req-5e272ad6-7bfc-49d3-b6bb-558caf67b4a8 service neutron] 10.222.0.74 "GET /v2.0/networks/e57f6e29-4637-455e-af9d-0975c45c995c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1815386 May 20 04:26:00.054603 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6350424d-0718-4e5b-948b-ded8293f840f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-7786687'] for subnet, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:00.098458 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-079e4f10-dc00-4c87-8089-c012e16ede75 req-b1266f24-18ba-444a-85a3-d9169c2bb6f2 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:00.099966 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-079e4f10-dc00-4c87-8089-c012e16ede75 req-b1266f24-18ba-444a-85a3-d9169c2bb6f2 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.0959568 May 20 04:26:00.194283 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-079e4f10-dc00-4c87-8089-c012e16ede75 req-8ba656b3-a2f4-4b38-97fa-c26a32895450 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0551541 May 20 04:26:00.207194 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2b630893-8e9d-43a5-8ca9-a4157ca79e3f req-430ed16c-51bb-4430-b71e-29201772ca2a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:00.208642 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b630893-8e9d-43a5-8ca9-a4157ca79e3f req-430ed16c-51bb-4430-b71e-29201772ca2a service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=39febf983bf24778a300f287dbeca829&device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 1096 time: 0.1078494 May 20 04:26:00.311852 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6350424d-0718-4e5b-948b-ded8293f840f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-7786632', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-1988925', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-1810229'] for subnet, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:00.316779 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-ee838674-9e9f-4e72-b5b4-fd48df802b5b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:00.351594 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6350424d-0718-4e5b-948b-ded8293f840f admin admin] Pushing event deleted for resources: {'Subnet': ['ID=0f918085-8d33-4e36-a59e-7f270a2b1874,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:00.354310 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-6350424d-0718-4e5b-948b-ded8293f840f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.038s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:00.362659 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6350424d-0718-4e5b-948b-ded8293f840f admin admin] Notify callbacks [] for subnet, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:00.369393 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-6350424d-0718-4e5b-948b-ded8293f840f admin admin] 10.222.0.39 "DELETE /v2.0/subnets/0f918085-8d33-4e36-a59e-7f270a2b1874 HTTP/1.1" status: 204 len: 168 time: 0.4969091 May 20 04:26:00.371755 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 38388) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:00.429336 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2b630893-8e9d-43a5-8ca9-a4157ca79e3f req-5a9dc2db-875f-40c5-892d-ccb4ac83d213 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:00.430950 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b630893-8e9d-43a5-8ca9-a4157ca79e3f req-5a9dc2db-875f-40c5-892d-ccb4ac83d213 service neutron] 10.222.0.74 "GET /v2.0/networks?id=e57f6e29-4637-455e-af9d-0975c45c995c HTTP/1.1" status: 200 len: 909 time: 0.2151401 May 20 04:26:00.513045 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b630893-8e9d-43a5-8ca9-a4157ca79e3f req-ec423b3e-7159-45c7-9c15-1006eccb25f7 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=b858096a-44b8-4537-a167-0e9c03347445 HTTP/1.1" status: 200 len: 217 time: 0.0740435 May 20 04:26:00.585443 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4218ac92-7d35-4774-88c0-3eb87f8b85d6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-593607'] for network, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:00.595712 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2b630893-8e9d-43a5-8ca9-a4157ca79e3f req-605d60b5-684e-4490-adca-04534d9f1d39 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:00.597236 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b630893-8e9d-43a5-8ca9-a4157ca79e3f req-605d60b5-684e-4490-adca-04534d9f1d39 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=49523a89-260b-4732-abff-507d74fed4c4 HTTP/1.1" status: 200 len: 847 time: 0.0721240 May 20 04:26:00.712559 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4218ac92-7d35-4774-88c0-3eb87f8b85d6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-7789108', 'neutron.services.segments.db._delete_segments_for_network-8760688958082'] for network, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:00.725958 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-4218ac92-7d35-4774-88c0-3eb87f8b85d6 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['54fc8d38-193b-49c4-b873-c5353723791f']}} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:26:00.733000 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2b630893-8e9d-43a5-8ca9-a4157ca79e3f req-cac5dd74-ca3b-487b-a0f4-a337a214d416 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:00.734554 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b630893-8e9d-43a5-8ca9-a4157ca79e3f req-cac5dd74-ca3b-487b-a0f4-a337a214d416 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=e57f6e29-4637-455e-af9d-0975c45c995c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1241035 May 20 04:26:00.739491 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-4218ac92-7d35-4774-88c0-3eb87f8b85d6 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '6fe99325-a72c-48bf-abc7-9bbcc4b753ff') {'for_net_delete': True} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:26:00.739784 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-4218ac92-7d35-4774-88c0-3eb87f8b85d6 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '6fe99325-a72c-48bf-abc7-9bbcc4b753ff') {} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:26:00.752049 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4218ac92-7d35-4774-88c0-3eb87f8b85d6 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8760688968331', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-853278'] for segment, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:00.769927 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4218ac92-7d35-4774-88c0-3eb87f8b85d6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:00.776852 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-4218ac92-7d35-4774-88c0-3eb87f8b85d6 admin admin] Releasing vlan 836 on physical network physnet1 to pool {{(pid=91644) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 20 04:26:00.777831 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4218ac92-7d35-4774-88c0-3eb87f8b85d6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2001055'] for segment, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: Traceback (most recent call last): May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: timer() May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: cb(*args, **kw) May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: return func(*args, **kwargs) May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: self._notify() May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: self.callback(batched_events) May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: event.method(event) May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: aggregate_id = self._get_aggregate_id(event.segment_id) May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: aggregate_uuid = self.p_client.list_aggregates( May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: return f(self, *a, **k) May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: return self._get(url).json() May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: return self._client.get(url, endpoint_filter=self._ks_filter, May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: return self.request(url, 'GET', **kwargs) May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: base_url = self.get_endpoint(auth, allow=allow, May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: auth = self._auth_required(auth, 'determine endpoint URL') May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 20 04:26:00.791704 dv-cinder-782405-9-iscsi neutron-server[91644]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 20 04:26:00.850854 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0509af2a-1f7a-42a2-8b44-849815c666f6 req-9d908cdd-e028-4174-b0a7-ef3f2b318214 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:00.852333 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0509af2a-1f7a-42a2-8b44-849815c666f6 req-9d908cdd-e028-4174-b0a7-ef3f2b318214 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.1825135 May 20 04:26:00.872329 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4218ac92-7d35-4774-88c0-3eb87f8b85d6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-7789117', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-8053321'] for network, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:00.876340 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:00.889321 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4218ac92-7d35-4774-88c0-3eb87f8b85d6 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:00.899637 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4218ac92-7d35-4774-88c0-3eb87f8b85d6 admin admin] Pushing event deleted for resources: {'Network': ['ID=54fc8d38-193b-49c4-b873-c5353723791f,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:00.902759 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-4218ac92-7d35-4774-88c0-3eb87f8b85d6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:00.909118 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-4218ac92-7d35-4774-88c0-3eb87f8b85d6 admin admin] 10.222.0.39 "DELETE /v2.0/networks/54fc8d38-193b-49c4-b873-c5353723791f HTTP/1.1" status: 204 len: 168 time: 0.5363441 May 20 04:26:00.913308 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b630893-8e9d-43a5-8ca9-a4157ca79e3f req-02947f1c-1063-48b5-9459-e8009e268aa9 service neutron] 10.222.0.74 "GET /v2.0/networks/e57f6e29-4637-455e-af9d-0975c45c995c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1704316 May 20 04:26:00.920033 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0509af2a-1f7a-42a2-8b44-849815c666f6 req-c746c5c5-5270-414c-b90f-74d508a84eb8 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0600104 May 20 04:26:00.967381 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 849868be-af51-4967-9a6c-93d23cecb450 no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:26:01.004378 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: 849868be-af51-4967-9a6c-93d23cecb450 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:26:01.004758 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 849868be-af51-4967-9a6c-93d23cecb450 not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:26:01.047582 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b630893-8e9d-43a5-8ca9-a4157ca79e3f req-5d9877c2-ea67-4040-8193-2b326785fb11 service neutron] 10.222.0.74 "GET /v2.0/networks/e57f6e29-4637-455e-af9d-0975c45c995c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1263328 May 20 04:26:01.097673 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 38398) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:01.149917 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-a612b7b6-dcf2-45ef-9721-1340f27eb017 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=51fcbc78284541db832e4780e76158af&name=default HTTP/1.1" status: 200 len: 2975 time: 0.0487759 May 20 04:26:01.150107 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 38404) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:01.210108 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-052b851a-ff7e-4d1e-a8cb-ea47194a9240 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:01.237154 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-052b851a-ff7e-4d1e-a8cb-ea47194a9240 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:01.292696 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-052b851a-ff7e-4d1e-a8cb-ea47194a9240 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:01.297940 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-81a6fe09-a3b3-49fe-add5-ef3b1242cec4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:01.310757 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-052b851a-ff7e-4d1e-a8cb-ea47194a9240 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:01.313545 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-052b851a-ff7e-4d1e-a8cb-ea47194a9240 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:01.321896 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-052b851a-ff7e-4d1e-a8cb-ea47194a9240 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:01.323026 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-052b851a-ff7e-4d1e-a8cb-ea47194a9240 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/7fc2375b-f1c9-4ad1-bb0c-1c2cb9970d87 HTTP/1.1" status: 204 len: 168 time: 0.1720331 May 20 04:26:01.395033 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5683ae31-1310-46be-a5f8-97d2fbd80427 req-a6996d21-1e9d-4bcd-9740-f68f0b39dfd0 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:01.396526 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5683ae31-1310-46be-a5f8-97d2fbd80427 req-a6996d21-1e9d-4bcd-9740-f68f0b39dfd0 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.0925100 May 20 04:26:01.442169 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5683ae31-1310-46be-a5f8-97d2fbd80427 req-8d7296d1-a636-434b-a988-4cc0e3665418 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0388200 May 20 04:26:02.108916 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dd3935cf-75ab-45cd-81d4-2c67fb19be3b req-eb30580a-0586-4d03-a378-fedf7ec212b3 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:02.110354 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-dd3935cf-75ab-45cd-81d4-2c67fb19be3b req-eb30580a-0586-4d03-a378-fedf7ec212b3 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0796230 May 20 04:26:02.157611 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-dd3935cf-75ab-45cd-81d4-2c67fb19be3b req-78bb1cab-f9ec-4d7d-a008-348c7c6a8b17 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0413170 May 20 04:26:02.632577 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-338e8ea5-28bb-4cde-9592-3312fa037bbb req-2ab3cdae-267d-470f-88e6-b0d76acde4e9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:02.634036 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-338e8ea5-28bb-4cde-9592-3312fa037bbb req-2ab3cdae-267d-470f-88e6-b0d76acde4e9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.0812125 May 20 04:26:02.678866 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-338e8ea5-28bb-4cde-9592-3312fa037bbb req-dd37e50a-49c7-4d8e-8ee1-d9a4a712f8db tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0389040 May 20 04:26:03.346105 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e1e9cc3a-cb11-4cf2-bc02-101b701db8e8 req-dca880f7-1698-493f-aa97-a0a528bcaed3 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:03.347718 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e1e9cc3a-cb11-4cf2-bc02-101b701db8e8 req-dca880f7-1698-493f-aa97-a0a528bcaed3 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.0846987 May 20 04:26:03.393703 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e1e9cc3a-cb11-4cf2-bc02-101b701db8e8 req-f05f4f13-d124-4d54-ae57-e699e779708c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0401628 May 20 04:26:03.448925 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 38484) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:03.591496 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-25c8ac88-97c6-490a-ae61-a431df424432 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:03.606374 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-25c8ac88-97c6-490a-ae61-a431df424432 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:03.610618 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-25c8ac88-97c6-490a-ae61-a431df424432 admin admin] Usage tracker for resource:security_group_rule and tenant:1f24f382324646039124051a9f3f5efe is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:03.613501 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-25c8ac88-97c6-490a-ae61-a431df424432 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:03.619543 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-25c8ac88-97c6-490a-ae61-a431df424432 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:03.707883 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-25c8ac88-97c6-490a-ae61-a431df424432 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:03.712077 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-25c8ac88-97c6-490a-ae61-a431df424432 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:03.719718 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-595032b4-d954-4afa-94b4-d5f6e21fc4d9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:03.745982 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-25c8ac88-97c6-490a-ae61-a431df424432 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=711809b1-6289-47f2-b20f-b7a41d4e71c9,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:03.750896 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-25c8ac88-97c6-490a-ae61-a431df424432 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.031s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:03.775802 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-25c8ac88-97c6-490a-ae61-a431df424432 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=1f24f382324646039124051a9f3f5efe&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3257303 May 20 04:26:03.776550 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 38492) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:03.875718 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b21ec863-b79d-40e9-a15b-52c1eb90558d req-a7725313-b7be-474f-b286-9cef958d4c02 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:03.877244 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b21ec863-b79d-40e9-a15b-52c1eb90558d req-a7725313-b7be-474f-b286-9cef958d4c02 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.74 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.1608062 May 20 04:26:03.959799 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-465e2de2-971a-4e19-a3c2-2c2e86e97936 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:04.034471 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-465e2de2-971a-4e19-a3c2-2c2e86e97936 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:04.072234 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9f96548c-83d5-444c-bd14-8575b76ee592 req-98849717-e141-483e-b3b8-760fe72f1320 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:04.073665 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9f96548c-83d5-444c-bd14-8575b76ee592 req-98849717-e141-483e-b3b8-760fe72f1320 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.2736495 May 20 04:26:04.148155 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9f96548c-83d5-444c-bd14-8575b76ee592 req-07fc2b92-cf44-4ecf-8cff-d0dbfa0ee6d3 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0618408 May 20 04:26:04.150887 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-465e2de2-971a-4e19-a3c2-2c2e86e97936 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:04.156715 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:04.166891 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-b21ec863-b79d-40e9-a15b-52c1eb90558d req-b5e899a5-d8c2-4a1b-bd29-078c0c29cae0 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Deleting port a8b0c56a-171d-40e9-beb7-c7b63fa031b9 {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:26:04.167231 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-b21ec863-b79d-40e9-a15b-52c1eb90558d req-b5e899a5-d8c2-4a1b-bd29-078c0c29cae0 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:04.185535 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-465e2de2-971a-4e19-a3c2-2c2e86e97936 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=711809b1-6289-47f2-b20f-b7a41d4e71c9,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:04.200303 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-465e2de2-971a-4e19-a3c2-2c2e86e97936 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:04.204998 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-465e2de2-971a-4e19-a3c2-2c2e86e97936 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:04.206088 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-465e2de2-971a-4e19-a3c2-2c2e86e97936 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/711809b1-6289-47f2-b20f-b7a41d4e71c9 HTTP/1.1" status: 204 len: 168 time: 0.4286325 May 20 04:26:04.390046 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-b21ec863-b79d-40e9-a15b-52c1eb90558d req-b5e899a5-d8c2-4a1b-bd29-078c0c29cae0 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] For port a8b0c56a-171d-40e9-beb7-c7b63fa031b9, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=a8b0c56a-171d-40e9-beb7-c7b63fa031b9,segment=NetworkSegment(0032d9a3-07bf-489f-bb9d-03ab3ff14f98),segment_id=0032d9a3-07bf-489f-bb9d-03ab3ff14f98)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:26:04.390437 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-b21ec863-b79d-40e9-a15b-52c1eb90558d req-b5e899a5-d8c2-4a1b-bd29-078c0c29cae0 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:04.498890 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 38504) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:04.513911 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-b21ec863-b79d-40e9-a15b-52c1eb90558d req-b5e899a5-d8c2-4a1b-bd29-078c0c29cae0 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Calling delete_port for a8b0c56a-171d-40e9-beb7-c7b63fa031b9 owned by compute:nova {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:26:04.516822 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-77babf16-6ab8-45b4-b32a-edf6cc347ed8 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:04.531434 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-77babf16-6ab8-45b4-b32a-edf6cc347ed8 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:04.535618 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-77babf16-6ab8-45b4-b32a-edf6cc347ed8 admin admin] Usage tracker for resource:security_group_rule and tenant:39673c28ed4e483f9fca8138c54cdcab is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:04.538402 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-77babf16-6ab8-45b4-b32a-edf6cc347ed8 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:04.543994 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-77babf16-6ab8-45b4-b32a-edf6cc347ed8 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:04.627854 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-77babf16-6ab8-45b4-b32a-edf6cc347ed8 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:04.629565 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b1cab637-f441-4ad4-b704-99cd7c8c72d7 req-8987c328-1e5e-4847-aeda-ca33b72a509f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:04.631169 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b1cab637-f441-4ad4-b704-99cd7c8c72d7 req-8987c328-1e5e-4847-aeda-ca33b72a509f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=0d234180-40a1-4229-990e-f796674056ce HTTP/1.1" status: 200 len: 946 time: 0.1240227 May 20 04:26:04.637976 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-77babf16-6ab8-45b4-b32a-edf6cc347ed8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:04.646780 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-27514652-fb14-4cb3-9f26-5c7459ad77cc admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:04.690593 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-77babf16-6ab8-45b4-b32a-edf6cc347ed8 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=39673c28ed4e483f9fca8138c54cdcab&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1881313 May 20 04:26:04.690827 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 38510) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:04.702487 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-77babf16-6ab8-45b4-b32a-edf6cc347ed8 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=cb8f5b26-6b24-454b-9774-9d0795dfc376,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:04.706921 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-77babf16-6ab8-45b4-b32a-edf6cc347ed8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.060s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:04.730814 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b1cab637-f441-4ad4-b704-99cd7c8c72d7 req-c6612364-cf93-4231-8546-61e4cbb818cd tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0934334 May 20 04:26:04.741031 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-b21ec863-b79d-40e9-a15b-52c1eb90558d req-b5e899a5-d8c2-4a1b-bd29-078c0c29cae0 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:04.747346 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-77004aef-ee56-4b0d-b795-e03d50b40230 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:04.752325 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['a8b0c56a-171d-40e9-beb7-c7b63fa031b9', '706277c6-cd8c-4146-ab91-ed9504446690'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:26:04.769247 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a4c784f4-67ae-410f-818e-c36b91fd7359 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:04.795887 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a4c784f4-67ae-410f-818e-c36b91fd7359 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:04.796165 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b5e899a5-d8c2-4a1b-bd29-078c0c29cae0 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Pushing event deleted for resources: {'Port': ['ID=a8b0c56a-171d-40e9-beb7-c7b63fa031b9,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:04.799018 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-b5e899a5-d8c2-4a1b-bd29-078c0c29cae0 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.052s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:04.855068 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a4c784f4-67ae-410f-818e-c36b91fd7359 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:04.860517 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-917cdfa3-6f49-4708-aa5d-da00916d70e8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:04.872348 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [req-b21ec863-b79d-40e9-a15b-52c1eb90558d req-b5e899a5-d8c2-4a1b-bd29-078c0c29cae0 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] No DVR routers for this DVR port a8b0c56a-171d-40e9-beb7-c7b63fa031b9 on host hv-cinder-78240 {{(pid=91643) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:26:04.872919 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a4c784f4-67ae-410f-818e-c36b91fd7359 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=cb8f5b26-6b24-454b-9774-9d0795dfc376,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:04.875198 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-b21ec863-b79d-40e9-a15b-52c1eb90558d req-b5e899a5-d8c2-4a1b-bd29-078c0c29cae0 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:04.877799 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-a4c784f4-67ae-410f-818e-c36b91fd7359 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:04.882173 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a4c784f4-67ae-410f-818e-c36b91fd7359 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:04.883314 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-a4c784f4-67ae-410f-818e-c36b91fd7359 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/cb8f5b26-6b24-454b-9774-9d0795dfc376 HTTP/1.1" status: 204 len: 168 time: 0.1916363 May 20 04:26:04.888342 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '0d234180-40a1-4229-990e-f796674056ce', 'name': 'network-vif-deleted', 'tag': 'a8b0c56a-171d-40e9-beb7-c7b63fa031b9'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:26:04.895079 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b21ec863-b79d-40e9-a15b-52c1eb90558d req-b5e899a5-d8c2-4a1b-bd29-078c0c29cae0 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.74 "DELETE /v2.0/ports/a8b0c56a-171d-40e9-beb7-c7b63fa031b9 HTTP/1.1" status: 204 len: 173 time: 1.0098269 May 20 04:26:04.937319 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '0d234180-40a1-4229-990e-f796674056ce', 'name': 'network-vif-deleted', 'tag': 'a8b0c56a-171d-40e9-beb7-c7b63fa031b9', 'status': 'completed', 'code': 200} May 20 04:26:05.340198 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3c699bf4-71bb-470a-a9f1-29f07ebffb75 req-44409bf6-f6ce-4879-a447-50bde833d8d7 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:05.341945 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3c699bf4-71bb-470a-a9f1-29f07ebffb75 req-44409bf6-f6ce-4879-a447-50bde833d8d7 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.0845561 May 20 04:26:05.387482 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3c699bf4-71bb-470a-a9f1-29f07ebffb75 req-91bde4e4-23ed-424a-a791-d86c66311984 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0394266 May 20 04:26:05.562122 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:06.199329 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e754b2f6-8fc4-4f8b-b460-acf1e6196b25 req-59198242-4804-46c2-8a31-e4bbaa1786e7 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:06.200038 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-e754b2f6-8fc4-4f8b-b460-acf1e6196b25 req-59198242-4804-46c2-8a31-e4bbaa1786e7 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Synchronizing usage tracker for tenant:bec56d8703cf47f3b7193b7d1f55396d on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:26:06.216476 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-e754b2f6-8fc4-4f8b-b460-acf1e6196b25 req-59198242-4804-46c2-8a31-e4bbaa1786e7 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Unset dirty status for tenant:bec56d8703cf47f3b7193b7d1f55396d on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:26:06.217848 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e754b2f6-8fc4-4f8b-b460-acf1e6196b25 req-59198242-4804-46c2-8a31-e4bbaa1786e7 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0989230 May 20 04:26:06.263148 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e754b2f6-8fc4-4f8b-b460-acf1e6196b25 req-614818e5-6508-4707-bd81-29aa3c7edd12 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0382769 May 20 04:26:06.579237 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eedfde52-509b-4f64-a6cf-6bbaa486d7cb req-b6d93e93-8b3a-43e4-b237-7eb4df014414 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:06.580734 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-eedfde52-509b-4f64-a6cf-6bbaa486d7cb req-b6d93e93-8b3a-43e4-b237-7eb4df014414 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.0834770 May 20 04:26:06.626666 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-eedfde52-509b-4f64-a6cf-6bbaa486d7cb req-b92b78f3-9927-4676-8c05-403b98149ab9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0400710 May 20 04:26:06.976974 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device a8b0c56a-171d-40e9-beb7-c7b63fa031b9 no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:26:07.015692 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: a8b0c56a-171d-40e9-beb7-c7b63fa031b9 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:26:07.015905 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device a8b0c56a-171d-40e9-beb7-c7b63fa031b9 not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:26:07.128124 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2262e2a0-3c2b-4e54-985e-3271b1d173c1 req-fbe77506-1621-4a46-adc9-de5849881e29 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0759029 May 20 04:26:07.223836 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2262e2a0-3c2b-4e54-985e-3271b1d173c1 req-c38e9014-71a1-4a5e-bca7-6d4ca650aa98 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:07.225262 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2262e2a0-3c2b-4e54-985e-3271b1d173c1 req-c38e9014-71a1-4a5e-bca7-6d4ca650aa98 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=9fafdf3a29024ac588e5b24719735677&device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 1098 time: 0.0727642 May 20 04:26:07.376074 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2262e2a0-3c2b-4e54-985e-3271b1d173c1 req-e543c928-39b2-4522-a280-1664ff9c1f4c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:07.377503 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2262e2a0-3c2b-4e54-985e-3271b1d173c1 req-e543c928-39b2-4522-a280-1664ff9c1f4c service neutron] 10.222.0.74 "GET /v2.0/networks?id=64bcea8a-6d35-406b-a823-fa1185008174 HTTP/1.1" status: 200 len: 918 time: 0.1280282 May 20 04:26:07.517453 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2262e2a0-3c2b-4e54-985e-3271b1d173c1 req-95a5c5e8-a6d0-46f4-80f0-45bed78ae691 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=535d5637-ae66-43cb-bd16-3543bd3b7a8b HTTP/1.1" status: 200 len: 1001 time: 0.1114421 May 20 04:26:07.578537 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-650a8dcc-2ee6-4ebd-a6d9-6b106f1ad9c8 req-c8828f1a-7132-40d3-bfed-5094691228b7 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:07.580176 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-650a8dcc-2ee6-4ebd-a6d9-6b106f1ad9c8 req-c8828f1a-7132-40d3-bfed-5094691228b7 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.2062995 May 20 04:26:07.608805 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2262e2a0-3c2b-4e54-985e-3271b1d173c1 req-70d31f05-1794-4bad-a3d1-1238d3eb7018 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:07.610147 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2262e2a0-3c2b-4e54-985e-3271b1d173c1 req-70d31f05-1794-4bad-a3d1-1238d3eb7018 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=203a3962-2282-4f51-ac92-c9695b5c0cfe HTTP/1.1" status: 200 len: 856 time: 0.0852480 May 20 04:26:07.668325 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-650a8dcc-2ee6-4ebd-a6d9-6b106f1ad9c8 req-f4120f7b-23fd-4820-8820-613c1d8f21db tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0815392 May 20 04:26:07.696585 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2262e2a0-3c2b-4e54-985e-3271b1d173c1 req-6f0e2b0c-1de7-4e44-962f-5f60ce5b74c6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:07.698170 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2262e2a0-3c2b-4e54-985e-3271b1d173c1 req-6f0e2b0c-1de7-4e44-962f-5f60ce5b74c6 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=64bcea8a-6d35-406b-a823-fa1185008174&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0782251 May 20 04:26:07.969213 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2262e2a0-3c2b-4e54-985e-3271b1d173c1 req-72804b2d-7298-400a-b5a4-21dd012fec3f service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2636230 May 20 04:26:07.984489 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0beb0606-6506-40c5-ac0e-3f6512ecf460 req-281712b5-b00d-48fd-b233-5b563062c8e1 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:07.985829 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0beb0606-6506-40c5-ac0e-3f6512ecf460 req-281712b5-b00d-48fd-b233-5b563062c8e1 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.2466354 May 20 04:26:08.116798 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0beb0606-6506-40c5-ac0e-3f6512ecf460 req-b58786d3-1014-4833-aac5-158b50cbffaa tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1139510 May 20 04:26:08.146931 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2262e2a0-3c2b-4e54-985e-3271b1d173c1 req-19d3766d-6f0f-4c66-8ca4-31b43d4ef1f8 service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1700094 May 20 04:26:08.870875 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8ded034d-11a4-41b2-8a78-b10dd8f7a21f req-0001fcb7-38a0-43aa-8c6a-fd3b3d81936c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:08.872504 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8ded034d-11a4-41b2-8a78-b10dd8f7a21f req-0001fcb7-38a0-43aa-8c6a-fd3b3d81936c tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0828130 May 20 04:26:08.920673 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8ded034d-11a4-41b2-8a78-b10dd8f7a21f req-29302dc4-2025-428b-9481-d8b2170b28eb tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0422139 May 20 04:26:08.988150 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b858096a-44b8-4537-a167-0e9c03347445 details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:26:09.121825 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b858096a-44b8-4537-a167-0e9c03347445, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b858096a-44b8-4537-a167-0e9c03347445,segment=NetworkSegment(c7d77442-5ac6-48cf-ab59-3f5962fc1e3e),segment_id=c7d77442-5ac6-48cf-ab59-3f5962fc1e3e)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:26:09.136080 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': 'b858096a-44b8-4537-a167-0e9c03347445', 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c', 'port_id': 'b858096a-44b8-4537-a167-0e9c03347445', 'mac_address': 'fa:16:3e:6c:52:6d', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 806, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '49523a89-260b-4732-abff-507d74fed4c4', 'ip_address': '172.20.0.5'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:26:09.190674 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:09.207621 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id b858096a-44b8-4537-a167-0e9c03347445 {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:26:09.225044 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b858096a-44b8-4537-a167-0e9c03347445, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b858096a-44b8-4537-a167-0e9c03347445,segment=NetworkSegment(c7d77442-5ac6-48cf-ab59-3f5962fc1e3e),segment_id=c7d77442-5ac6-48cf-ab59-3f5962fc1e3e)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:26:09.231124 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:09.234961 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-e730247e-f308-4816-8b4d-1425b012cfe5 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:09.308635 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=b858096a-44b8-4537-a167-0e9c03347445,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:09.316878 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:09.317215 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:09.318199 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:09.318199 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:09.318199 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:09.319117 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:09.319502 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5b8ad116-1933-4c60-9197-4670f2d1c0db req-0831024d-1236-4fb0-838e-1405378940d4 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:09.319607 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:09.319607 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:09.319607 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:09.319840 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:09.321241 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:09.321377 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5b8ad116-1933-4c60-9197-4670f2d1c0db req-0831024d-1236-4fb0-838e-1405378940d4 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 944 time: 0.0834358 May 20 04:26:09.321462 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:09.325068 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.090s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:09.380928 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5b8ad116-1933-4c60-9197-4670f2d1c0db req-5c180885-3d04-46c1-bb29-259d11774c6e tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0520010 May 20 04:26:09.765370 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['b858096a-44b8-4537-a167-0e9c03347445'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:26:10.108454 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f97a4563-cfda-43b9-8846-a76081654cbd req-201a00c2-e21d-457b-ad1b-853d98ac64a3 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:10.109877 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f97a4563-cfda-43b9-8846-a76081654cbd req-201a00c2-e21d-457b-ad1b-853d98ac64a3 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0843196 May 20 04:26:10.155131 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f97a4563-cfda-43b9-8846-a76081654cbd req-f9e32af4-7afc-4595-8395-8854acdbbbfa tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0393641 May 20 04:26:10.574849 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f9736068-9ef0-4043-a8df-6151d864c3e3 req-e6bae09d-f153-4604-876c-9c6de0ad91e5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:10.576339 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f9736068-9ef0-4043-a8df-6151d864c3e3 req-e6bae09d-f153-4604-876c-9c6de0ad91e5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 944 time: 0.0795627 May 20 04:26:10.625021 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f9736068-9ef0-4043-a8df-6151d864c3e3 req-a1b36f28-1c4e-4321-87a1-f6b8bbc61c05 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0429082 May 20 04:26:10.997666 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b858096a-44b8-4537-a167-0e9c03347445 up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:26:11.068220 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port b858096a-44b8-4537-a167-0e9c03347445 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:26:11.073061 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port b858096a-44b8-4537-a167-0e9c03347445 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:26:11.073444 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:11.205247 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:11.241804 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b858096a-44b8-4537-a167-0e9c03347445, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b858096a-44b8-4537-a167-0e9c03347445,segment=NetworkSegment(c7d77442-5ac6-48cf-ab59-3f5962fc1e3e),segment_id=c7d77442-5ac6-48cf-ab59-3f5962fc1e3e)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:26:11.243653 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '30d5e70e-9acd-400c-83be-01301f56a5ff', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b858096a-44b8-4537-a167-0e9c03347445'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:26:11.249872 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:11.251908 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:11.325118 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=b858096a-44b8-4537-a167-0e9c03347445,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:11.328194 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.076s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:11.382376 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c9e830f3-f8ed-446b-bb02-e09f20dbd253 req-5f0153ed-aec2-4a26-afa5-b1c89e6f9404 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:11.384434 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c9e830f3-f8ed-446b-bb02-e09f20dbd253 req-5f0153ed-aec2-4a26-afa5-b1c89e6f9404 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0844591 May 20 04:26:11.431402 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c9e830f3-f8ed-446b-bb02-e09f20dbd253 req-24643382-e893-4840-8b3a-e9ddb1d32275 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0406835 May 20 04:26:11.480847 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '30d5e70e-9acd-400c-83be-01301f56a5ff', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b858096a-44b8-4537-a167-0e9c03347445', 'code': 200} May 20 04:26:11.613234 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-51704d73-1c43-4a13-91f3-23591ea57590 req-91aa3d6e-f816-44fa-bd75-a21150688fc3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:11.614760 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-51704d73-1c43-4a13-91f3-23591ea57590 req-91aa3d6e-f816-44fa-bd75-a21150688fc3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.0838389 May 20 04:26:11.664216 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-51704d73-1c43-4a13-91f3-23591ea57590 req-8ab1a8b8-8212-4f41-9fb7-2d11672c71d3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0433838 May 20 04:26:11.881871 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5b68c814-e746-400e-b1e9-341d8dbb4d82 req-2cc7c7d5-52aa-4011-8f81-c7d14a3c5596 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:11.883370 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5b68c814-e746-400e-b1e9-341d8dbb4d82 req-2cc7c7d5-52aa-4011-8f81-c7d14a3c5596 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.1360912 May 20 04:26:11.887829 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-178fc2c9-ef1f-4b9b-bfa1-eb73a9702159 req-7774596a-6725-4cd9-bb6b-5f4f88188ca6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:11.889261 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-178fc2c9-ef1f-4b9b-bfa1-eb73a9702159 req-7774596a-6725-4cd9-bb6b-5f4f88188ca6 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 1098 time: 0.1112244 May 20 04:26:11.953859 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5b68c814-e746-400e-b1e9-341d8dbb4d82 req-14378e6e-d0ef-4a92-a730-bcd9c3c0fee7 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0637491 May 20 04:26:11.957170 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-178fc2c9-ef1f-4b9b-bfa1-eb73a9702159 req-a49f7139-a918-4371-ba7f-76ee8f00c3a3 service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0546975 May 20 04:26:12.605778 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-3bee2d6b-80bd-42eb-9918-e0abf1eec8ee None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:12.635790 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cc05856d-23b2-46bd-a612-30cd395a624d req-de92e70f-9869-4e6b-b2da-bda9a25f20fd tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:12.637518 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cc05856d-23b2-46bd-a612-30cd395a624d req-de92e70f-9869-4e6b-b2da-bda9a25f20fd tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.0948539 May 20 04:26:12.688524 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cc05856d-23b2-46bd-a612-30cd395a624d req-4eb30538-7d26-4817-b42b-e3f7c73992f0 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0445786 May 20 04:26:12.998764 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a63954fc-1757-45e2-80d2-5223317a143e req-7695d665-57ba-483c-81ca-d70d4b417f06 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:13.000247 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a63954fc-1757-45e2-80d2-5223317a143e req-7695d665-57ba-483c-81ca-d70d4b417f06 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.0818753 May 20 04:26:13.046459 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a63954fc-1757-45e2-80d2-5223317a143e req-2f5fdbb3-698d-4477-a52e-8729f48e4794 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0402431 May 20 04:26:13.236240 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1f2513cb-ddcc-4ca3-af61-30feebd4427d req-37e21804-a462-47fa-9577-09c13489f25f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:13.237545 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1f2513cb-ddcc-4ca3-af61-30feebd4427d req-37e21804-a462-47fa-9577-09c13489f25f tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.74 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1373627 May 20 04:26:13.251242 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-94ddc78e-cc3a-4840-a7e0-226199732c17 req-dd3bc1f0-a92f-4b70-8452-1e8cd21cbe66 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:13.252744 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-94ddc78e-cc3a-4840-a7e0-226199732c17 req-dd3bc1f0-a92f-4b70-8452-1e8cd21cbe66 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.1084316 May 20 04:26:13.385137 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-94ddc78e-cc3a-4840-a7e0-226199732c17 req-4a9fc362-f239-4fa7-b02e-9ea08c725a59 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1002455 May 20 04:26:13.426898 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-1f2513cb-ddcc-4ca3-af61-30feebd4427d req-ac065659-71c5-49c0-ac75-d87375539664 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Deleting port 706277c6-cd8c-4146-ab91-ed9504446690 {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:26:13.427265 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-1f2513cb-ddcc-4ca3-af61-30feebd4427d req-ac065659-71c5-49c0-ac75-d87375539664 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:13.568415 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-1f2513cb-ddcc-4ca3-af61-30feebd4427d req-ac065659-71c5-49c0-ac75-d87375539664 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] For port 706277c6-cd8c-4146-ab91-ed9504446690, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=706277c6-cd8c-4146-ab91-ed9504446690,segment=NetworkSegment(0032d9a3-07bf-489f-bb9d-03ab3ff14f98),segment_id=0032d9a3-07bf-489f-bb9d-03ab3ff14f98)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:26:13.568731 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-1f2513cb-ddcc-4ca3-af61-30feebd4427d req-ac065659-71c5-49c0-ac75-d87375539664 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:13.765095 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-1f2513cb-ddcc-4ca3-af61-30feebd4427d req-ac065659-71c5-49c0-ac75-d87375539664 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Calling delete_port for 706277c6-cd8c-4146-ab91-ed9504446690 owned by compute:nova {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:26:13.933342 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-1f2513cb-ddcc-4ca3-af61-30feebd4427d req-ac065659-71c5-49c0-ac75-d87375539664 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:13.939801 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:13.945215 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['706277c6-cd8c-4146-ab91-ed9504446690'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:26:13.993226 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e29c834f-9cb7-4cf8-9069-612b1f65c31a req-7db53dce-d1d5-4f26-9769-f5d091349617 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:13.996542 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e29c834f-9cb7-4cf8-9069-612b1f65c31a req-7db53dce-d1d5-4f26-9769-f5d091349617 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/ports?device_id=bbf6ae7d-8ff5-4b69-adef-47220d6c5c93 HTTP/1.1" status: 200 len: 945 time: 0.1915035 May 20 04:26:13.999591 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ac065659-71c5-49c0-ac75-d87375539664 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Pushing event deleted for resources: {'Port': ['ID=706277c6-cd8c-4146-ab91-ed9504446690,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:14.028512 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-ac065659-71c5-49c0-ac75-d87375539664 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.089s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:14.094377 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [req-1f2513cb-ddcc-4ca3-af61-30feebd4427d req-ac065659-71c5-49c0-ac75-d87375539664 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] No DVR routers for this DVR port 706277c6-cd8c-4146-ab91-ed9504446690 on host hv-cinder-78240 {{(pid=91643) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:26:14.102901 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-1f2513cb-ddcc-4ca3-af61-30feebd4427d req-ac065659-71c5-49c0-ac75-d87375539664 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:14.105264 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'bbf6ae7d-8ff5-4b69-adef-47220d6c5c93', 'name': 'network-vif-deleted', 'tag': '706277c6-cd8c-4146-ab91-ed9504446690'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:26:14.126766 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e29c834f-9cb7-4cf8-9069-612b1f65c31a req-9a16fe0e-3ec4-4cdf-8964-72e6b2835a58 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.39 "GET /v2.0/security-groups?id=54593a4a-1755-4771-9f21-b88328299a32&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1250269 May 20 04:26:14.133201 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1f2513cb-ddcc-4ca3-af61-30feebd4427d req-ac065659-71c5-49c0-ac75-d87375539664 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] 10.222.0.74 "DELETE /v2.0/ports/706277c6-cd8c-4146-ab91-ed9504446690 HTTP/1.1" status: 204 len: 173 time: 0.8795547 May 20 04:26:14.245722 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d7fb5236-b9ba-4e30-93f2-6222fc94adfe req-f2e601fc-5aea-49c9-b908-af945931a594 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:14.247155 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d7fb5236-b9ba-4e30-93f2-6222fc94adfe req-f2e601fc-5aea-49c9-b908-af945931a594 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.0864940 May 20 04:26:14.292494 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d7fb5236-b9ba-4e30-93f2-6222fc94adfe req-33ebcc0d-e912-4ba1-bbad-065e4c97841b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0392456 May 20 04:26:14.331865 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'bbf6ae7d-8ff5-4b69-adef-47220d6c5c93', 'name': 'network-vif-deleted', 'tag': '706277c6-cd8c-4146-ab91-ed9504446690', 'status': 'completed', 'code': 200} May 20 04:26:14.579575 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ad29d3e4-cbcf-45d9-9744-4e4c48d161e0 req-f2244af6-c44c-496a-aa4f-260c9142f876 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:14.581005 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ad29d3e4-cbcf-45d9-9744-4e4c48d161e0 req-f2244af6-c44c-496a-aa4f-260c9142f876 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.0879889 May 20 04:26:14.627725 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ad29d3e4-cbcf-45d9-9744-4e4c48d161e0 req-3fba922b-a504-4fcb-b6a3-f249aa016aa4 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0402758 May 20 04:26:14.824752 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-06f1fbf9-5fcf-42bc-82d5-d09b73367ea0 req-99a098fd-92a8-4de4-8561-cc17c0aec7df tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:14.826165 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-06f1fbf9-5fcf-42bc-82d5-d09b73367ea0 req-99a098fd-92a8-4de4-8561-cc17c0aec7df tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.0896449 May 20 04:26:14.873940 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-06f1fbf9-5fcf-42bc-82d5-d09b73367ea0 req-27fd9879-8082-4266-b662-e8613003e1ad tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0386930 May 20 04:26:14.901606 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 38734) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:14.963748 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:15.097565 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-9e33ca72-3978-49f2-93db-920b68d08eab None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:15.155751 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-d2899523-15e6-4e85-8062-58eee8bd921f tempest-TestShelveInstance-1178194002 tempest-TestShelveInstance-1178194002-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:15.156460 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-d2899523-15e6-4e85-8062-58eee8bd921f tempest-TestShelveInstance-1178194002 tempest-TestShelveInstance-1178194002-project-admin] Synchronizing usage tracker for tenant:03fbb7d8cfaa4c2ab40ef558c109e377 on resource:port {{(pid=91644) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:26:15.174947 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-d2899523-15e6-4e85-8062-58eee8bd921f tempest-TestShelveInstance-1178194002 tempest-TestShelveInstance-1178194002-project-admin] Unset dirty status for tenant:03fbb7d8cfaa4c2ab40ef558c109e377 on resource:port {{(pid=91644) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:26:15.179568 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-d2899523-15e6-4e85-8062-58eee8bd921f tempest-TestShelveInstance-1178194002 tempest-TestShelveInstance-1178194002-project-admin] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 1093 time: 0.2769389 May 20 04:26:15.179762 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 38746) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:15.185140 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Request body: {'floatingip': {'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'port_id': 'b858096a-44b8-4537-a167-0e9c03347445', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'fixed_ip_address': '172.20.0.5'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:26:15.191035 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:15.195458 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Usage tracker for resource:floatingip and tenant:39febf983bf24778a300f287dbeca829 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:15.198331 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:15.204995 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:15.217316 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for floatingip, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:15.237110 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 38750) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:15.751667 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-492978b9-57b1-4b8c-95bc-e7b922e46412 req-be54534f-49b6-462c-a97d-8bace2da925f tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:15.752721 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-492978b9-57b1-4b8c-95bc-e7b922e46412 req-be54534f-49b6-462c-a97d-8bace2da925f tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.3065069 May 20 04:26:15.885170 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:15.985944 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-492978b9-57b1-4b8c-95bc-e7b922e46412 req-a0b7dccd-194c-4d39-b108-a757abee73b3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.2089875 May 20 04:26:16.092936 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Allocated IP 172.27.0.129 (712ce8c4-8feb-4719-bf13-e06992125cd9/e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5/7a1ffe9b-0e93-4df0-b2ca-608471a67ed8) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:26:16.179450 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:16.293520 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:16.311569 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b44e7aff-acb4-47b4-b69f-bed9f2333fc4 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-8003145'] for router_interface, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:16.470743 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:16.476807 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-93299e6b-f51b-4971-8862-eec8a4c587df service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:16.547556 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:26:16.592193 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:26:16.601021 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:26:16.676790 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Pushing event updated for resources: {'Port': ['ID=7a1ffe9b-0e93-4df0-b2ca-608471a67ed8,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:16.687080 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.210s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:17.016306 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 706277c6-cd8c-4146-ab91-ed9504446690 no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:26:17.056903 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: 706277c6-cd8c-4146-ab91-ed9504446690 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:26:17.057267 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 706277c6-cd8c-4146-ab91-ed9504446690 not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:26:17.379119 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-b44e7aff-acb4-47b4-b69f-bed9f2333fc4 admin admin] Deleting port 29b05a85-1060-41b1-aff1-48c1783a174f {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:26:17.379748 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b44e7aff-acb4-47b4-b69f-bed9f2333fc4 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:17.560214 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [None req-b44e7aff-acb4-47b4-b69f-bed9f2333fc4 admin admin] For port 29b05a85-1060-41b1-aff1-48c1783a174f, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=29b05a85-1060-41b1-aff1-48c1783a174f,segment=NetworkSegment(0032d9a3-07bf-489f-bb9d-03ab3ff14f98),segment_id=0032d9a3-07bf-489f-bb9d-03ab3ff14f98)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:26:17.560574 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b44e7aff-acb4-47b4-b69f-bed9f2333fc4 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:17.577733 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for floatingip, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:17.617449 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-88573f7b-5c0b-420f-94eb-4a2eee608082 req-6a48afef-9eff-4475-9a9e-bc4c5ee54cc1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:17.618908 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-88573f7b-5c0b-420f-94eb-4a2eee608082 req-6a48afef-9eff-4475-9a9e-bc4c5ee54cc1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.5085139 May 20 04:26:17.750909 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-b44e7aff-acb4-47b4-b69f-bed9f2333fc4 admin admin] Calling delete_port for 29b05a85-1060-41b1-aff1-48c1783a174f owned by network:router_interface {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:26:17.818886 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:17.821648 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-88573f7b-5c0b-420f-94eb-4a2eee608082 req-d6d864ed-185f-48b3-963f-9778ac70ece9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1467583 May 20 04:26:17.858827 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:18.285084 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for port, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:18.288847 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b44e7aff-acb4-47b4-b69f-bed9f2333fc4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:18.293270 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:18.322634 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:18.342181 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b44e7aff-acb4-47b4-b69f-bed9f2333fc4 admin admin] Pushing event deleted for resources: {'Port': ['ID=29b05a85-1060-41b1-aff1-48c1783a174f,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:18.377259 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-b44e7aff-acb4-47b4-b69f-bed9f2333fc4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:18.378919 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-af1b9814-c5fa-47a6-87bb-72763e738c11 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.047s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:18.523637 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Pushing event updated for resources: {'Port': ['ID=7a1ffe9b-0e93-4df0-b2ca-608471a67ed8,revision_number=2']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:18.530793 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.152s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:18.536290 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-8003251'] for floatingip, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:18.969896 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.l3_db [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Floating IP c8510538-1bcc-4560-92c1-c5a6e93ee028 associated. External IP: 172.27.0.129, port: b858096a-44b8-4537-a167-0e9c03347445. May 20 04:26:19.022398 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b44e7aff-acb4-47b4-b69f-bed9f2333fc4 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-7924922', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-8054745'] for router_interface, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:19.336633 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5b6d815-5b85-475e-95a7-0e1c846cd952 req-c4392540-a104-4a8e-9e97-dd75e69416fb tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:19.338084 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b5b6d815-5b85-475e-95a7-0e1c846cd952 req-c4392540-a104-4a8e-9e97-dd75e69416fb tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.4034002 May 20 04:26:19.532271 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b5b6d815-5b85-475e-95a7-0e1c846cd952 req-b5196757-891d-427e-a676-79a2e269e238 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1457176 May 20 04:26:19.697158 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Router 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91643) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:26:19.702791 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-b44e7aff-acb4-47b4-b69f-bed9f2333fc4 admin admin] Router be02e160-6134-4b60-b833-5a659cc82c44 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91643) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:26:20.144496 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:26:20.150906 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-b44e7aff-acb4-47b4-b69f-bed9f2333fc4 admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:26:20.153999 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4b338324-dd68-4699-afa1-5d8f5c738e3c None None] Sync routers for ids ['5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'] in dv-cinder-782405-9-iscsi {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:26:20.164438 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-52036c26-36a6-488a-8dc8-20eaa5c0f553 None None] Sync routers for ids ['be02e160-6134-4b60-b833-5a659cc82c44'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:26:20.167434 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4b338324-dd68-4699-afa1-5d8f5c738e3c None None] Routers {'5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:26:20.168379 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-4b338324-dd68-4699-afa1-5d8f5c738e3c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 8, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 8, "log_agent_heartbeats": false, "routers": 8},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:26:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:26:20.178196 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-52036c26-36a6-488a-8dc8-20eaa5c0f553 None None] Routers {'be02e160-6134-4b60-b833-5a659cc82c44'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:26:20.178840 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-52036c26-36a6-488a-8dc8-20eaa5c0f553 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 8, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 8, "log_agent_heartbeats": false, "routers": 8},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:26:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['be02e160-6134-4b60-b833-5a659cc82c44'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:26:20.182736 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-b44e7aff-acb4-47b4-b69f-bed9f2333fc4 admin admin] 10.222.0.39 "PUT /v2.0/routers/be02e160-6134-4b60-b833-5a659cc82c44/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 4.9445930 May 20 04:26:20.184113 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:20.185560 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 38812) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:20.255918 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 996 time: 5.0750258 May 20 04:26:20.256852 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '30d5e70e-9acd-400c-83be-01301f56a5ff', 'tag': 'b858096a-44b8-4537-a167-0e9c03347445'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:26:20.310759 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '30d5e70e-9acd-400c-83be-01301f56a5ff', 'tag': 'b858096a-44b8-4537-a167-0e9c03347445', 'status': 'completed', 'code': 200} May 20 04:26:20.375193 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-52036c26-36a6-488a-8dc8-20eaa5c0f553 None None] Set router be02e160-6134-4b60-b833-5a659cc82c44 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:26:20.378067 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eec55cf8-a8b1-41d3-8903-cb5b3a355bcf admin admin] Notify callbacks [] for router, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:20.400010 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-4b338324-dd68-4699-afa1-5d8f5c738e3c None None] Set router 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:26:20.417733 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-21c12344-96ec-4618-97ef-5cc24af92d7f req-ecde2fde-364c-4eea-b35a-4d7b1a822756 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:20.419361 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-21c12344-96ec-4618-97ef-5cc24af92d7f req-ecde2fde-364c-4eea-b35a-4d7b1a822756 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=39febf983bf24778a300f287dbeca829&device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 1098 time: 0.0752175 May 20 04:26:20.560425 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-21c12344-96ec-4618-97ef-5cc24af92d7f req-09686c59-a517-41cc-9794-aa2c75c59ce1 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:20.562769 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-21c12344-96ec-4618-97ef-5cc24af92d7f req-09686c59-a517-41cc-9794-aa2c75c59ce1 service neutron] 10.222.0.74 "GET /v2.0/networks?id=e57f6e29-4637-455e-af9d-0975c45c995c HTTP/1.1" status: 200 len: 909 time: 0.1365106 May 20 04:26:20.605559 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eec55cf8-a8b1-41d3-8903-cb5b3a355bcf admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:20.681510 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-21c12344-96ec-4618-97ef-5cc24af92d7f req-66699c2d-e4aa-4baf-914c-3af46522c70b service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=b858096a-44b8-4537-a167-0e9c03347445 HTTP/1.1" status: 200 len: 999 time: 0.1107733 May 20 04:26:20.707435 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-52036c26-36a6-488a-8dc8-20eaa5c0f553 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'be02e160-6134-4b60-b833-5a659cc82c44', 'name': 'tempest-ServerRescueNegativeTestJSON-1026520542-router', 'tenant_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.120'}], 'enable_snat': True}, 'gw_port_id': '68d41f86-efee-45dc-988a-b22c7179ec88', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:43Z', 'updated_at': '2021-05-20T04:26:18Z', 'revision_number': 5, 'project_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'gw_port': {'id': '68d41f86-efee-45dc-988a-b22c7179ec88', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:65:dc:1e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'be02e160-6134-4b60-b833-5a659cc82c44', 'device_owner': 'network:router_gateway', 'standard_attr_id': 83, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.120', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:46Z', 'updated_at': '2021-05-20T04:21:57Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:26:20.712615 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-52036c26-36a6-488a-8dc8-20eaa5c0f553 None None] Checking router: be02e160-6134-4b60-b833-5a659cc82c44 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:26:20.716416 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-eec55cf8-a8b1-41d3-8903-cb5b3a355bcf admin admin] Deleting port 68d41f86-efee-45dc-988a-b22c7179ec88 {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:26:20.717295 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eec55cf8-a8b1-41d3-8903-cb5b3a355bcf admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:20.724277 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-52036c26-36a6-488a-8dc8-20eaa5c0f553 None None] Routers {'be02e160-6134-4b60-b833-5a659cc82c44'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:26:20.725149 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-52036c26-36a6-488a-8dc8-20eaa5c0f553 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 8, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 8, "log_agent_heartbeats": false, "routers": 8},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:26:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['be02e160-6134-4b60-b833-5a659cc82c44'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:26:20.772889 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-4b338324-dd68-4699-afa1-5d8f5c738e3c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'name': 'tempest-TestShelveInstance-251001111-router', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100'}], 'enable_snat': True}, 'gw_port_id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:08Z', 'updated_at': '2021-05-20T04:25:15Z', 'revision_number': 4, 'project_id': '39febf983bf24778a300f287dbeca829', 'gw_port': {'id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:d3:05:8a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:10Z', 'updated_at': '2021-05-20T04:25:16Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'c8510538-1bcc-4560-92c1-c5a6e93ee028', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'floating_ip_address': '172.27.0.129', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'port_id': 'b858096a-44b8-4537-a167-0e9c03347445', 'fixed_ip_address': '172.20.0.5', 'status': 'DOWN', 'standard_attr_id': 392, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c', 'mac_address': 'fa:16:3e:6c:52:6d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '30d5e70e-9acd-400c-83be-01301f56a5ff', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:26:17Z', 'updated_at': '2021-05-20T04:26:17Z', 'revision_number': 0, 'project_id': '39febf983bf24778a300f287dbeca829', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'c7df749b-cc66-4529-a666-5c9e0a997ff8', 'name': '', 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'mac_address': 'fa:16:3e:8f:27:b0', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_interface', 'standard_attr_id': 355, 'fixed_ips': [{'subnet_id': '49523a89-260b-4732-abff-507d74fed4c4', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:14Z', 'updated_at': '2021-05-20T04:25:22Z', 'revision_number': 5, 'project_id': '39febf983bf24778a300f287dbeca829', 'subnets': [{'id': '49523a89-260b-4732-abff-507d74fed4c4', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:26:20.777542 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4b338324-dd68-4699-afa1-5d8f5c738e3c None None] Checking router: 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 for host: dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:26:20.786263 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4b338324-dd68-4699-afa1-5d8f5c738e3c None None] Routers {'5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:26:20.787094 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-4b338324-dd68-4699-afa1-5d8f5c738e3c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 8, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 8, "log_agent_heartbeats": false, "routers": 8},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:26:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:26:20.805339 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-21c12344-96ec-4618-97ef-5cc24af92d7f req-7ef3c89e-3bc9-4ff4-953e-e3103eb4524b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:20.806801 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-21c12344-96ec-4618-97ef-5cc24af92d7f req-7ef3c89e-3bc9-4ff4-953e-e3103eb4524b service neutron] 10.222.0.74 "GET /v2.0/subnets?id=49523a89-260b-4732-abff-507d74fed4c4 HTTP/1.1" status: 200 len: 847 time: 0.1189950 May 20 04:26:20.851970 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [None req-eec55cf8-a8b1-41d3-8903-cb5b3a355bcf admin admin] For port 68d41f86-efee-45dc-988a-b22c7179ec88, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=68d41f86-efee-45dc-988a-b22c7179ec88,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:26:20.852503 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eec55cf8-a8b1-41d3-8903-cb5b3a355bcf admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:20.902542 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-52036c26-36a6-488a-8dc8-20eaa5c0f553 None None] Set router be02e160-6134-4b60-b833-5a659cc82c44 gateway port host: None {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:26:20.904935 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-549fca17-7666-4068-9342-1f636e81f4e0 req-71e1e6e9-1c85-46e1-9e04-d52ee12963db tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:20.912307 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-549fca17-7666-4068-9342-1f636e81f4e0 req-71e1e6e9-1c85-46e1-9e04-d52ee12963db tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1814189 May 20 04:26:20.963118 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-4b338324-dd68-4699-afa1-5d8f5c738e3c None None] Set router 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:26:20.971158 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-549fca17-7666-4068-9342-1f636e81f4e0 req-d4978b9a-9dce-4154-83f4-3294c1075a95 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0578783 May 20 04:26:20.990259 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-21c12344-96ec-4618-97ef-5cc24af92d7f req-7a43dd21-eed5-4486-9b0f-ddc64b157511 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:20.991958 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-21c12344-96ec-4618-97ef-5cc24af92d7f req-7a43dd21-eed5-4486-9b0f-ddc64b157511 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=e57f6e29-4637-455e-af9d-0975c45c995c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1746604 May 20 04:26:21.044736 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-52036c26-36a6-488a-8dc8-20eaa5c0f553 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'be02e160-6134-4b60-b833-5a659cc82c44', 'name': 'tempest-ServerRescueNegativeTestJSON-1026520542-router', 'tenant_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:43Z', 'updated_at': '2021-05-20T04:26:20Z', 'revision_number': 6, 'project_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'gw_port_host': None}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:26:21.064453 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-52036c26-36a6-488a-8dc8-20eaa5c0f553 None None] The sync data for ids ['be02e160-6134-4b60-b833-5a659cc82c44'] in dv-cinder-782405-9-iscsi is: [{'id': 'be02e160-6134-4b60-b833-5a659cc82c44', 'name': 'tempest-ServerRescueNegativeTestJSON-1026520542-router', 'tenant_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:43Z', 'updated_at': '2021-05-20T04:26:20Z', 'revision_number': 6, 'project_id': 'bec56d8703cf47f3b7193b7d1f55396d', 'gw_port_host': None}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:26:21.167765 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-21c12344-96ec-4618-97ef-5cc24af92d7f req-258da1c8-1398-4d1d-84d0-cff144e6ddb4 service neutron] 10.222.0.74 "GET /v2.0/networks/e57f6e29-4637-455e-af9d-0975c45c995c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1679037 May 20 04:26:21.214091 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-eec55cf8-a8b1-41d3-8903-cb5b3a355bcf admin admin] Calling delete_port for 68d41f86-efee-45dc-988a-b22c7179ec88 owned by network:router_gateway {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:26:21.318741 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-21c12344-96ec-4618-97ef-5cc24af92d7f req-a27ab909-96a0-4342-a33c-d88b27e017c2 service neutron] 10.222.0.74 "GET /v2.0/networks/e57f6e29-4637-455e-af9d-0975c45c995c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1402025 May 20 04:26:21.383707 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-4b338324-dd68-4699-afa1-5d8f5c738e3c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'name': 'tempest-TestShelveInstance-251001111-router', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100'}], 'enable_snat': True}, 'gw_port_id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:08Z', 'updated_at': '2021-05-20T04:25:15Z', 'revision_number': 4, 'project_id': '39febf983bf24778a300f287dbeca829', 'gw_port': {'id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:d3:05:8a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:10Z', 'updated_at': '2021-05-20T04:25:16Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'c8510538-1bcc-4560-92c1-c5a6e93ee028', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'floating_ip_address': '172.27.0.129', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'port_id': 'b858096a-44b8-4537-a167-0e9c03347445', 'fixed_ip_address': '172.20.0.5', 'status': 'DOWN', 'standard_attr_id': 392, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c', 'mac_address': 'fa:16:3e:6c:52:6d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '30d5e70e-9acd-400c-83be-01301f56a5ff', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:26:17Z', 'updated_at': '2021-05-20T04:26:17Z', 'revision_number': 0, 'project_id': '39febf983bf24778a300f287dbeca829', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'c7df749b-cc66-4529-a666-5c9e0a997ff8', 'name': '', 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'mac_address': 'fa:16:3e:8f:27:b0', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_interface', 'standard_attr_id': 355, 'fixed_ips': [{'subnet_id': '49523a89-260b-4732-abff-507d74fed4c4', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:14Z', 'updated_at': '2021-05-20T04:25:22Z', 'revision_number': 5, 'project_id': '39febf983bf24778a300f287dbeca829', 'subnets': [{'id': '49523a89-260b-4732-abff-507d74fed4c4', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:26:21.387742 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4b338324-dd68-4699-afa1-5d8f5c738e3c None None] The sync data for ids ['5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'] in dv-cinder-782405-9-iscsi is: [{'id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'name': 'tempest-TestShelveInstance-251001111-router', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100'}], 'enable_snat': True}, 'gw_port_id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:08Z', 'updated_at': '2021-05-20T04:25:15Z', 'revision_number': 4, 'project_id': '39febf983bf24778a300f287dbeca829', 'gw_port': {'id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:d3:05:8a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:10Z', 'updated_at': '2021-05-20T04:25:16Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'c8510538-1bcc-4560-92c1-c5a6e93ee028', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'floating_ip_address': '172.27.0.129', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'port_id': 'b858096a-44b8-4537-a167-0e9c03347445', 'fixed_ip_address': '172.20.0.5', 'status': 'DOWN', 'standard_attr_id': 392, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c', 'mac_address': 'fa:16:3e:6c:52:6d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '30d5e70e-9acd-400c-83be-01301f56a5ff', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:26:17Z', 'updated_at': '2021-05-20T04:26:17Z', 'revision_number': 0, 'project_id': '39febf983bf24778a300f287dbeca829', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'c7df749b-cc66-4529-a666-5c9e0a997ff8', 'name': '', 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'mac_address': 'fa:16:3e:8f:27:b0', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_interface', 'standard_attr_id': 355, 'fixed_ips': [{'subnet_id': '49523a89-260b-4732-abff-507d74fed4c4', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:14Z', 'updated_at': '2021-05-20T04:25:22Z', 'revision_number': 5, 'project_id': '39febf983bf24778a300f287dbeca829', 'subnets': [{'id': '49523a89-260b-4732-abff-507d74fed4c4', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:26:21.508481 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eec55cf8-a8b1-41d3-8903-cb5b3a355bcf admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:21.513334 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:21.591077 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-eec55cf8-a8b1-41d3-8903-cb5b3a355bcf admin admin] Pushing event deleted for resources: {'Port': ['ID=68d41f86-efee-45dc-988a-b22c7179ec88,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:21.593846 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-eec55cf8-a8b1-41d3-8903-cb5b3a355bcf admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:21.684788 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eec55cf8-a8b1-41d3-8903-cb5b3a355bcf admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-8053346'] for router_gateway, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:21.692146 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-524c7fa9-4d2b-4390-aca4-87dded999ca9 None None] New status for floating IP c8510538-1bcc-4560-92c1-c5a6e93ee028: ACTIVE {{(pid=91646) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 20 04:26:21.840735 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 68d41f86-efee-45dc-988a-b22c7179ec88 no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:26:21.869369 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eec55cf8-a8b1-41d3-8903-cb5b3a355bcf admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-8071641', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-260698'] for router, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:21.871817 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eec55cf8-a8b1-41d3-8903-cb5b3a355bcf admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:21.883272 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: 68d41f86-efee-45dc-988a-b22c7179ec88 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:26:21.883931 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 68d41f86-efee-45dc-988a-b22c7179ec88 not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:26:21.884345 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 29b05a85-1060-41b1-aff1-48c1783a174f no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:26:21.907688 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:26:21.920372 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: 29b05a85-1060-41b1-aff1-48c1783a174f {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:26:21.920842 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 29b05a85-1060-41b1-aff1-48c1783a174f not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:26:21.964754 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eec55cf8-a8b1-41d3-8903-cb5b3a355bcf admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-8056629'] for router, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:21.965258 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-eec55cf8-a8b1-41d3-8903-cb5b3a355bcf admin admin] Fanout notify agent at l3_agent the message router_deleted on router be02e160-6134-4b60-b833-5a659cc82c44 {{(pid=91644) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 20 04:26:21.979926 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eec55cf8-a8b1-41d3-8903-cb5b3a355bcf admin admin] Notify callbacks [] for router, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:21.983632 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-eec55cf8-a8b1-41d3-8903-cb5b3a355bcf admin admin] 10.222.0.39 "DELETE /v2.0/routers/be02e160-6134-4b60-b833-5a659cc82c44 HTTP/1.1" status: 204 len: 168 time: 1.7961910 May 20 04:26:21.983893 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 38844) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:22.057909 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_v2 [None req-1e204bf6-94c1-450b-a254-0641f88ad1c5 admin admin] Deleting subnet b25ae9c8-e11d-45ee-bde2-332cc1538602 {{(pid=91643) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 20 04:26:22.062144 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-1e204bf6-94c1-450b-a254-0641f88ad1c5 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:22.298719 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-1e204bf6-94c1-450b-a254-0641f88ad1c5 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-1472754'] for subnet, precommit_delete_associations {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:22.361553 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-1e204bf6-94c1-450b-a254-0641f88ad1c5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-7786687'] for subnet, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:22.411047 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-22d27cc0-044c-4e83-907e-ff41c4072283 req-7d4d8a6a-bc4a-4ad6-b923-acc66bd4d9d3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:22.412540 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-22d27cc0-044c-4e83-907e-ff41c4072283 req-7d4d8a6a-bc4a-4ad6-b923-acc66bd4d9d3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.3114109 May 20 04:26:22.542404 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-22d27cc0-044c-4e83-907e-ff41c4072283 req-4391715c-8b60-44e2-a477-2e1077e8c58d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1230669 May 20 04:26:22.659414 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-1e204bf6-94c1-450b-a254-0641f88ad1c5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-7786632', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2050499', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2172490'] for subnet, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:22.664057 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-a253eddc-9e40-4c7f-b691-30b352c841ac admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:22.697873 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1e204bf6-94c1-450b-a254-0641f88ad1c5 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=b25ae9c8-e11d-45ee-bde2-332cc1538602,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:22.699794 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-1e204bf6-94c1-450b-a254-0641f88ad1c5 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:22.702657 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-1e204bf6-94c1-450b-a254-0641f88ad1c5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.038s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:22.706435 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-1e204bf6-94c1-450b-a254-0641f88ad1c5 admin admin] 10.222.0.39 "DELETE /v2.0/subnets/b25ae9c8-e11d-45ee-bde2-332cc1538602 HTTP/1.1" status: 204 len: 168 time: 0.7215300 May 20 04:26:22.709025 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 38854) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:22.814527 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7c54a2e7-6ad9-4afa-8a05-2709739bf3f5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-593607'] for network, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:22.940943 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7c54a2e7-6ad9-4afa-8a05-2709739bf3f5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-7789108', 'neutron.services.segments.db._delete_segments_for_network-8760688958082'] for network, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:22.954067 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-7c54a2e7-6ad9-4afa-8a05-2709739bf3f5 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['36b37213-8439-447d-9eba-a78baf7c698e']}} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:26:22.967461 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-7c54a2e7-6ad9-4afa-8a05-2709739bf3f5 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '0032d9a3-07bf-489f-bb9d-03ab3ff14f98') {'for_net_delete': True} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:26:22.967781 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-7c54a2e7-6ad9-4afa-8a05-2709739bf3f5 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '0032d9a3-07bf-489f-bb9d-03ab3ff14f98') {} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:26:22.979359 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7c54a2e7-6ad9-4afa-8a05-2709739bf3f5 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8760688968331', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-1472741'] for segment, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:22.994833 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7c54a2e7-6ad9-4afa-8a05-2709739bf3f5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:23.000965 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-7c54a2e7-6ad9-4afa-8a05-2709739bf3f5 admin admin] Releasing vlan 813 on physical network physnet1 to pool {{(pid=91643) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 20 04:26:23.001419 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7c54a2e7-6ad9-4afa-8a05-2709739bf3f5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2070625'] for segment, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: Traceback (most recent call last): May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: timer() May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: cb(*args, **kw) May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: return func(*args, **kwargs) May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: self._notify() May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: self.callback(batched_events) May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: event.method(event) May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: aggregate_id = self._get_aggregate_id(event.segment_id) May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: aggregate_uuid = self.p_client.list_aggregates( May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: return f(self, *a, **k) May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: return self._get(url).json() May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: return self._client.get(url, endpoint_filter=self._ks_filter, May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: return self.request(url, 'GET', **kwargs) May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: base_url = self.get_endpoint(auth, allow=allow, May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: auth = self._auth_required(auth, 'determine endpoint URL') May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 20 04:26:23.018310 dv-cinder-782405-9-iscsi neutron-server[91643]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 20 04:26:23.099330 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7c54a2e7-6ad9-4afa-8a05-2709739bf3f5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-7789117', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-8053321'] for network, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:23.103802 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-702afb13-8551-4828-a20a-543f3552efcc admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:23.119926 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7c54a2e7-6ad9-4afa-8a05-2709739bf3f5 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:23.126525 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7c54a2e7-6ad9-4afa-8a05-2709739bf3f5 admin admin] Pushing event deleted for resources: {'Network': ['ID=36b37213-8439-447d-9eba-a78baf7c698e,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:23.129319 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-7c54a2e7-6ad9-4afa-8a05-2709739bf3f5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:23.141113 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-7c54a2e7-6ad9-4afa-8a05-2709739bf3f5 admin admin] 10.222.0.39 "DELETE /v2.0/networks/36b37213-8439-447d-9eba-a78baf7c698e HTTP/1.1" status: 204 len: 168 time: 0.4309723 May 20 04:26:23.334367 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 38884) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:23.390691 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-8fb0842d-c02a-4f09-b40d-7e14e710e5e9 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=bec56d8703cf47f3b7193b7d1f55396d&name=default HTTP/1.1" status: 200 len: 2975 time: 0.0548422 May 20 04:26:23.393260 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 38890) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:23.438926 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 38894) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:23.452513 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-61797c5b-1dcb-46a3-a935-4eedd57ed2c9 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:23.479346 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-61797c5b-1dcb-46a3-a935-4eedd57ed2c9 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:23.531969 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-61797c5b-1dcb-46a3-a935-4eedd57ed2c9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:23.536831 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:23.550270 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-61797c5b-1dcb-46a3-a935-4eedd57ed2c9 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=54593a4a-1755-4771-9f21-b88328299a32,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:23.553056 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-61797c5b-1dcb-46a3-a935-4eedd57ed2c9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:23.555341 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4b224c26-4ff6-462f-92fb-1c7eb49ecb78 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:23.561648 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-61797c5b-1dcb-46a3-a935-4eedd57ed2c9 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:23.565138 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-61797c5b-1dcb-46a3-a935-4eedd57ed2c9 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/54593a4a-1755-4771-9f21-b88328299a32 HTTP/1.1" status: 204 len: 168 time: 0.1685383 May 20 04:26:23.577140 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-4b224c26-4ff6-462f-92fb-1c7eb49ecb78 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:23.581230 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-4b224c26-4ff6-462f-92fb-1c7eb49ecb78 admin admin] Usage tracker for resource:security_group_rule and tenant:0614e1b68895474aa67d7014f292e6d0 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:23.584277 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-4b224c26-4ff6-462f-92fb-1c7eb49ecb78 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:23.590649 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-4b224c26-4ff6-462f-92fb-1c7eb49ecb78 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:23.676659 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4b224c26-4ff6-462f-92fb-1c7eb49ecb78 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:23.681842 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4b224c26-4ff6-462f-92fb-1c7eb49ecb78 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:23.689255 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-fbc095e1-e940-4daf-bfcc-10e600e5a9e8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:23.730353 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4b224c26-4ff6-462f-92fb-1c7eb49ecb78 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=0ec05afb-493e-4ad7-a401-3aa59395e220,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:23.736984 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-4b224c26-4ff6-462f-92fb-1c7eb49ecb78 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=0614e1b68895474aa67d7014f292e6d0&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2945743 May 20 04:26:23.736984 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-4b224c26-4ff6-462f-92fb-1c7eb49ecb78 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:23.737254 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 38906) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:23.755239 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05d9b43e-2b4f-4b13-9f19-92929c44f8c6 req-db5ae764-226b-4976-9f62-cbc181fa998d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:23.763716 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-05d9b43e-2b4f-4b13-9f19-92929c44f8c6 req-db5ae764-226b-4976-9f62-cbc181fa998d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1037462 May 20 04:26:23.798693 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d6f98c60-a416-40d9-b9a6-8b7f7c7d2cff admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:23.806180 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-05d9b43e-2b4f-4b13-9f19-92929c44f8c6 req-0da3244a-c21d-4202-9158-796a1236e8ed tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0414064 May 20 04:26:23.825447 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d6f98c60-a416-40d9-b9a6-8b7f7c7d2cff admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:23.880297 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d6f98c60-a416-40d9-b9a6-8b7f7c7d2cff admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:23.886767 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-54d13a09-0311-47b3-ab5c-c42f3a4e16df admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:23.898748 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d6f98c60-a416-40d9-b9a6-8b7f7c7d2cff admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=0ec05afb-493e-4ad7-a401-3aa59395e220,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:23.904450 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d6f98c60-a416-40d9-b9a6-8b7f7c7d2cff admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:23.906768 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d6f98c60-a416-40d9-b9a6-8b7f7c7d2cff admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:23.908065 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-d6f98c60-a416-40d9-b9a6-8b7f7c7d2cff admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/0ec05afb-493e-4ad7-a401-3aa59395e220 HTTP/1.1" status: 204 len: 168 time: 0.1699626 May 20 04:26:24.201473 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 38928) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:24.218934 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-51fc048c-83a8-41f0-92b3-bedf34d4be55 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:24.233038 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-51fc048c-83a8-41f0-92b3-bedf34d4be55 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:24.237010 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-51fc048c-83a8-41f0-92b3-bedf34d4be55 admin admin] Usage tracker for resource:security_group_rule and tenant:09a3d9657654453fbf37b9d93cfdf6f5 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:24.239846 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-51fc048c-83a8-41f0-92b3-bedf34d4be55 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:24.245659 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-51fc048c-83a8-41f0-92b3-bedf34d4be55 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:24.325559 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-51fc048c-83a8-41f0-92b3-bedf34d4be55 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:24.329813 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-51fc048c-83a8-41f0-92b3-bedf34d4be55 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:24.338427 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e0cb0242-9851-4985-8bef-cd7964266488 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:24.376594 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-51fc048c-83a8-41f0-92b3-bedf34d4be55 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=09a3d9657654453fbf37b9d93cfdf6f5&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1739409 May 20 04:26:24.379103 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-51fc048c-83a8-41f0-92b3-bedf34d4be55 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=7977d706-24ff-4c26-b1f5-4354f391abfd,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:24.379362 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 38936) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:24.382038 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-51fc048c-83a8-41f0-92b3-bedf34d4be55 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:24.440088 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d37dd146-40c8-4530-82e9-7f4b30095805 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:24.465903 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d37dd146-40c8-4530-82e9-7f4b30095805 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:24.523061 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d37dd146-40c8-4530-82e9-7f4b30095805 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:24.529555 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-5657f594-f480-4bd7-b0d6-49b82791f43f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:24.542442 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d37dd146-40c8-4530-82e9-7f4b30095805 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=7977d706-24ff-4c26-b1f5-4354f391abfd,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:24.544744 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-d37dd146-40c8-4530-82e9-7f4b30095805 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:24.545874 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d37dd146-40c8-4530-82e9-7f4b30095805 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:24.549905 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-d37dd146-40c8-4530-82e9-7f4b30095805 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/7977d706-24ff-4c26-b1f5-4354f391abfd HTTP/1.1" status: 204 len: 168 time: 0.1666398 May 20 04:26:25.014524 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e872a797-964e-4947-b027-8e04ba9895b0 req-1ff32544-4a8c-4d65-9f7c-4273ae9e1035 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:25.019661 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e872a797-964e-4947-b027-8e04ba9895b0 req-1ff32544-4a8c-4d65-9f7c-4273ae9e1035 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.0923922 May 20 04:26:25.068173 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e872a797-964e-4947-b027-8e04ba9895b0 req-0005552b-431f-4ad3-a3d2-49cd7c5f81c2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0457132 May 20 04:26:25.094609 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 535d5637-ae66-43cb-bd16-3543bd3b7a8b details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:26:25.251529 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 535d5637-ae66-43cb-bd16-3543bd3b7a8b, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=535d5637-ae66-43cb-bd16-3543bd3b7a8b,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:26:25.267308 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': '535d5637-ae66-43cb-bd16-3543bd3b7a8b', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'port_id': '535d5637-ae66-43cb-bd16-3543bd3b7a8b', 'mac_address': 'fa:16:3e:ea:90:d5', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 847, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.6'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:26:25.324132 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:25.340613 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: ACTIVE current_port_status: BUILD port_id 535d5637-ae66-43cb-bd16-3543bd3b7a8b {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:26:25.356665 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 535d5637-ae66-43cb-bd16-3543bd3b7a8b, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=535d5637-ae66-43cb-bd16-3543bd3b7a8b,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:26:25.361613 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:25.363373 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:25.434658 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=535d5637-ae66-43cb-bd16-3543bd3b7a8b,revision_number=6']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:25.437628 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:25.473808 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['535d5637-ae66-43cb-bd16-3543bd3b7a8b'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:26:25.584679 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 39000) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:25.721904 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6422b44d-e479-410a-acab-89dfd0579690 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:25.736218 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-6422b44d-e479-410a-acab-89dfd0579690 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:25.740362 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-6422b44d-e479-410a-acab-89dfd0579690 admin admin] Usage tracker for resource:security_group_rule and tenant:9e1044ada7ea4448b6f6328751c0ff2e is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:25.743235 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-6422b44d-e479-410a-acab-89dfd0579690 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:25.749122 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-6422b44d-e479-410a-acab-89dfd0579690 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:25.840901 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6422b44d-e479-410a-acab-89dfd0579690 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:25.846148 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6422b44d-e479-410a-acab-89dfd0579690 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:25.854442 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-58148a96-e9d6-494e-b4f4-c61576684f71 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:25.895693 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-6422b44d-e479-410a-acab-89dfd0579690 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=9e1044ada7ea4448b6f6328751c0ff2e&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3092220 May 20 04:26:25.897217 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6422b44d-e479-410a-acab-89dfd0579690 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f4f2ad3b-50eb-4238-988f-944c849adc1f,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:25.898557 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 39014) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:25.900201 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-6422b44d-e479-410a-acab-89dfd0579690 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:25.963433 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c0748f8c-ee43-4533-b6ef-4a70716d53eb admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:25.989181 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c0748f8c-ee43-4533-b6ef-4a70716d53eb admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:26.042308 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c0748f8c-ee43-4533-b6ef-4a70716d53eb admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:26.047124 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-9d7e2ca2-0be3-4d4c-942d-e86b1d4133e0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:26.061738 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c0748f8c-ee43-4533-b6ef-4a70716d53eb admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f4f2ad3b-50eb-4238-988f-944c849adc1f,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:26.065697 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-c0748f8c-ee43-4533-b6ef-4a70716d53eb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:26.067649 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c0748f8c-ee43-4533-b6ef-4a70716d53eb admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:26.068737 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-c0748f8c-ee43-4533-b6ef-4a70716d53eb admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/f4f2ad3b-50eb-4238-988f-944c849adc1f HTTP/1.1" status: 204 len: 168 time: 0.1681645 May 20 04:26:26.265802 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f0502176-a262-4e23-bf98-a470b66ae5d9 req-7a9cb708-a090-4e0c-b516-3e2b968eda0a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:26.267490 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f0502176-a262-4e23-bf98-a470b66ae5d9 req-7a9cb708-a090-4e0c-b516-3e2b968eda0a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 944 time: 0.0831375 May 20 04:26:26.313950 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f0502176-a262-4e23-bf98-a470b66ae5d9 req-330ec96e-fbf7-4240-83c8-bd6ef3229354 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0393407 May 20 04:26:26.446267 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 39032) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:26.463644 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a3ec278d-fab6-439e-9230-44b916f3bb21 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:26.477815 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-a3ec278d-fab6-439e-9230-44b916f3bb21 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:26.481631 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-a3ec278d-fab6-439e-9230-44b916f3bb21 admin admin] Usage tracker for resource:security_group_rule and tenant:37f2e090def5462bbd151010f0ea3ce3 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:26.484484 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-a3ec278d-fab6-439e-9230-44b916f3bb21 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:26.490240 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-a3ec278d-fab6-439e-9230-44b916f3bb21 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:26.580972 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a3ec278d-fab6-439e-9230-44b916f3bb21 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:26.586172 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a3ec278d-fab6-439e-9230-44b916f3bb21 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:26.602153 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:26.638351 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-a3ec278d-fab6-439e-9230-44b916f3bb21 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=37f2e090def5462bbd151010f0ea3ce3&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1909876 May 20 04:26:26.641032 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 39040) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:26.642416 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a3ec278d-fab6-439e-9230-44b916f3bb21 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=2306d093-bfb9-4fdd-b22f-f54f545b296e,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:26.653398 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-a3ec278d-fab6-439e-9230-44b916f3bb21 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:26.703107 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-20542b25-396f-460c-ab64-9d8ed132d3c5 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:26.728802 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-20542b25-396f-460c-ab64-9d8ed132d3c5 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:26.792408 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-20542b25-396f-460c-ab64-9d8ed132d3c5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:26.796151 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-735b54fd-fca9-4a2c-87d6-a329e70019b5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:26.806764 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-20542b25-396f-460c-ab64-9d8ed132d3c5 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=2306d093-bfb9-4fdd-b22f-f54f545b296e,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:26.808806 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-20542b25-396f-460c-ab64-9d8ed132d3c5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.013s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:26.817387 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-20542b25-396f-460c-ab64-9d8ed132d3c5 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:26.818620 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-20542b25-396f-460c-ab64-9d8ed132d3c5 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/2306d093-bfb9-4fdd-b22f-f54f545b296e HTTP/1.1" status: 204 len: 168 time: 0.1764059 May 20 04:26:27.092148 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 535d5637-ae66-43cb-bd16-3543bd3b7a8b up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:26:27.161987 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port 535d5637-ae66-43cb-bd16-3543bd3b7a8b triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:26:27.162373 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:27.280565 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:27.307334 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'c2ca950f-78ff-4e44-9231-e8e19f4aee5d', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '535d5637-ae66-43cb-bd16-3543bd3b7a8b'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:26:27.317493 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 535d5637-ae66-43cb-bd16-3543bd3b7a8b, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=535d5637-ae66-43cb-bd16-3543bd3b7a8b,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:26:27.321741 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:27.323779 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:27.358766 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'c2ca950f-78ff-4e44-9231-e8e19f4aee5d', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '535d5637-ae66-43cb-bd16-3543bd3b7a8b', 'code': 200} May 20 04:26:27.390227 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=535d5637-ae66-43cb-bd16-3543bd3b7a8b,revision_number=7']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:27.393351 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.070s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:27.518639 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ddd88c90-f6cc-46e1-bc29-f50c9d2928e7 req-7ea8b094-0521-4e17-b277-f7b245933ddd tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:27.520252 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ddd88c90-f6cc-46e1-bc29-f50c9d2928e7 req-7ea8b094-0521-4e17-b277-f7b245933ddd tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.0862110 May 20 04:26:27.565060 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ddd88c90-f6cc-46e1-bc29-f50c9d2928e7 req-16473075-25f5-4105-a821-4ce59f1e2525 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0389354 May 20 04:26:28.571579 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 39114) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:28.733289 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e5be7a3b-f21e-47ae-a06e-d05909b668f6 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:28.756125 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-e5be7a3b-f21e-47ae-a06e-d05909b668f6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:28.761463 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-e5be7a3b-f21e-47ae-a06e-d05909b668f6 admin admin] Usage tracker for resource:security_group_rule and tenant:0b6be7dba9b64d2cb02670420cdc4756 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:28.765053 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-e5be7a3b-f21e-47ae-a06e-d05909b668f6 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:28.772116 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-e5be7a3b-f21e-47ae-a06e-d05909b668f6 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:28.808141 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a23b3598-b0e1-4fc8-8490-40be390be4cc req-ae7e82db-e2bb-46ca-8d49-ef9e3870fc04 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:28.809587 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a23b3598-b0e1-4fc8-8490-40be390be4cc req-ae7e82db-e2bb-46ca-8d49-ef9e3870fc04 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1368248 May 20 04:26:28.867695 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a23b3598-b0e1-4fc8-8490-40be390be4cc req-d73dab42-c56c-4c14-ab85-c31099887a41 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0520105 May 20 04:26:28.903812 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e5be7a3b-f21e-47ae-a06e-d05909b668f6 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:28.910101 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e5be7a3b-f21e-47ae-a06e-d05909b668f6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:28.920145 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-738b0e7d-f542-4d42-a53c-eef12f5aca28 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:28.970815 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-e5be7a3b-f21e-47ae-a06e-d05909b668f6 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=0b6be7dba9b64d2cb02670420cdc4756&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3979001 May 20 04:26:28.971952 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e5be7a3b-f21e-47ae-a06e-d05909b668f6 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=dae05660-b39c-45a8-b9a5-6a3597f42473,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:28.973720 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 39122) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:28.974959 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-e5be7a3b-f21e-47ae-a06e-d05909b668f6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:29.034058 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-de07e88e-fbf2-484e-be96-9714626b5f7c admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:29.059384 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-de07e88e-fbf2-484e-be96-9714626b5f7c admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:29.114963 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-de07e88e-fbf2-484e-be96-9714626b5f7c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:29.122352 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-052b851a-ff7e-4d1e-a8cb-ea47194a9240 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:29.140640 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-de07e88e-fbf2-484e-be96-9714626b5f7c admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=dae05660-b39c-45a8-b9a5-6a3597f42473,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:29.143001 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-de07e88e-fbf2-484e-be96-9714626b5f7c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.021s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:29.146670 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-de07e88e-fbf2-484e-be96-9714626b5f7c admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:29.150734 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-de07e88e-fbf2-484e-be96-9714626b5f7c admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/dae05660-b39c-45a8-b9a5-6a3597f42473 HTTP/1.1" status: 204 len: 168 time: 0.1753399 May 20 04:26:29.444618 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 39142) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:29.472075 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9a7d990d-e61d-4565-91e7-2fea33fa92f8 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:29.486991 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-9a7d990d-e61d-4565-91e7-2fea33fa92f8 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:29.491018 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-9a7d990d-e61d-4565-91e7-2fea33fa92f8 admin admin] Usage tracker for resource:security_group_rule and tenant:d89db4f9ee9a4088b4fefc2fceaaaad0 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:29.493913 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-9a7d990d-e61d-4565-91e7-2fea33fa92f8 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:29.499794 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-9a7d990d-e61d-4565-91e7-2fea33fa92f8 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:29.589269 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9a7d990d-e61d-4565-91e7-2fea33fa92f8 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:29.594158 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9a7d990d-e61d-4565-91e7-2fea33fa92f8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:29.603446 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-707cc159-8df8-4bc9-9eef-2f124c5a7a97 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:29.727580 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-9a7d990d-e61d-4565-91e7-2fea33fa92f8 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=d89db4f9ee9a4088b4fefc2fceaaaad0&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2818470 May 20 04:26:29.728447 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9a7d990d-e61d-4565-91e7-2fea33fa92f8 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=a8586bcb-6f54-40d6-9e34-349fd8afeb9f,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:29.730442 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 39152) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:29.731475 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-9a7d990d-e61d-4565-91e7-2fea33fa92f8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.128s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:29.802640 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9c0683ef-dc4f-42c5-aa84-f40f8badd25d admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:29.829040 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9c0683ef-dc4f-42c5-aa84-f40f8badd25d admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:29.884613 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9c0683ef-dc4f-42c5-aa84-f40f8badd25d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:29.889699 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-762e6639-b8ea-4715-bcdf-12d9eaf7578c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:29.909621 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9c0683ef-dc4f-42c5-aa84-f40f8badd25d admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=a8586bcb-6f54-40d6-9e34-349fd8afeb9f,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:29.912578 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9c0683ef-dc4f-42c5-aa84-f40f8badd25d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:29.933323 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9c0683ef-dc4f-42c5-aa84-f40f8badd25d admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:29.934902 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-9c0683ef-dc4f-42c5-aa84-f40f8badd25d admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/a8586bcb-6f54-40d6-9e34-349fd8afeb9f HTTP/1.1" status: 204 len: 168 time: 0.2035146 May 20 04:26:30.185278 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ec2fc5cb-4daa-4f6a-81f0-6feade75be34 req-8b12810a-81ab-4145-b797-218855ef2fc8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:30.186847 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ec2fc5cb-4daa-4f6a-81f0-6feade75be34 req-8b12810a-81ab-4145-b797-218855ef2fc8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.2001722 May 20 04:26:30.235132 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ec2fc5cb-4daa-4f6a-81f0-6feade75be34 req-b3f38024-c1b2-4e0f-a162-875acd7f1eba tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0425515 May 20 04:26:30.389330 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 39182) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:30.557165 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3f44364d-ba24-4615-a326-ff5666cb612a admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:30.573558 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-3f44364d-ba24-4615-a326-ff5666cb612a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:30.579576 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-3f44364d-ba24-4615-a326-ff5666cb612a admin admin] Usage tracker for resource:security_group_rule and tenant:9955cdc9844545168e863baa7e6d13c7 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:30.582646 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-3f44364d-ba24-4615-a326-ff5666cb612a admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:30.588757 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-3f44364d-ba24-4615-a326-ff5666cb612a admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:30.675624 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3f44364d-ba24-4615-a326-ff5666cb612a admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:30.681320 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3f44364d-ba24-4615-a326-ff5666cb612a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:30.693517 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-465e2de2-971a-4e19-a3c2-2c2e86e97936 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:30.732461 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-3f44364d-ba24-4615-a326-ff5666cb612a admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=9955cdc9844545168e863baa7e6d13c7&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3419170 May 20 04:26:30.734664 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3f44364d-ba24-4615-a326-ff5666cb612a admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=4a466eff-fbf0-4509-9a6c-a1eae73d01b2,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:30.735206 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 39200) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:30.743047 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-3f44364d-ba24-4615-a326-ff5666cb612a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:30.798205 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-abe4ad8b-a054-499c-bb0d-60c87a7120f5 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:30.824969 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-abe4ad8b-a054-499c-bb0d-60c87a7120f5 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:30.878422 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-abe4ad8b-a054-499c-bb0d-60c87a7120f5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:30.883766 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-50771e98-6b94-4ef3-a415-888b2767ab9e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:30.897368 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-abe4ad8b-a054-499c-bb0d-60c87a7120f5 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=4a466eff-fbf0-4509-9a6c-a1eae73d01b2,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:30.905931 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-abe4ad8b-a054-499c-bb0d-60c87a7120f5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.022s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:30.907168 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-abe4ad8b-a054-499c-bb0d-60c87a7120f5 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:30.911763 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-abe4ad8b-a054-499c-bb0d-60c87a7120f5 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/4a466eff-fbf0-4509-9a6c-a1eae73d01b2 HTTP/1.1" status: 204 len: 168 time: 0.1754115 May 20 04:26:31.237874 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 39230) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:31.260661 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-36da6ca3-a044-4e91-a4f8-c37b0b0d5c57 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:31.277748 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-36da6ca3-a044-4e91-a4f8-c37b0b0d5c57 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:31.282335 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-36da6ca3-a044-4e91-a4f8-c37b0b0d5c57 admin admin] Usage tracker for resource:security_group_rule and tenant:bb84e7d822df4aa1b3afda3d08bfd11d is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:31.285416 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-36da6ca3-a044-4e91-a4f8-c37b0b0d5c57 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:31.308818 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-36da6ca3-a044-4e91-a4f8-c37b0b0d5c57 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:31.490563 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-36da6ca3-a044-4e91-a4f8-c37b0b0d5c57 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:31.497673 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-36da6ca3-a044-4e91-a4f8-c37b0b0d5c57 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:31.507478 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-ad1194b0-df1b-4a95-9763-6e3a03418c59 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:31.548997 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-82d95d53-45c7-451d-92ff-0cd6aaabb6a1 req-f158a3d9-450c-487e-98d6-b0ee0270d23a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:31.550475 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-82d95d53-45c7-451d-92ff-0cd6aaabb6a1 req-f158a3d9-450c-487e-98d6-b0ee0270d23a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1874628 May 20 04:26:31.578993 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-36da6ca3-a044-4e91-a4f8-c37b0b0d5c57 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=bb84e7d822df4aa1b3afda3d08bfd11d&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3396928 May 20 04:26:31.581589 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-36da6ca3-a044-4e91-a4f8-c37b0b0d5c57 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=3ae35b95-076f-4240-9c8f-7d63b8db9b31,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:31.581874 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 39254) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:31.595942 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-36da6ca3-a044-4e91-a4f8-c37b0b0d5c57 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.088s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:31.622741 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-82d95d53-45c7-451d-92ff-0cd6aaabb6a1 req-f1f1e08d-0a3e-4aec-8ad6-e674210caefc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0636046 May 20 04:26:31.670582 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eb911ea4-f94f-47df-a048-51b3f0dbfea8 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:31.700069 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eb911ea4-f94f-47df-a048-51b3f0dbfea8 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:31.779200 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eb911ea4-f94f-47df-a048-51b3f0dbfea8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:31.791803 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-9a7d990d-e61d-4565-91e7-2fea33fa92f8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:31.804787 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-eb911ea4-f94f-47df-a048-51b3f0dbfea8 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=3ae35b95-076f-4240-9c8f-7d63b8db9b31,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:31.824307 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-eb911ea4-f94f-47df-a048-51b3f0dbfea8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:31.834819 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eb911ea4-f94f-47df-a048-51b3f0dbfea8 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:31.836784 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-eb911ea4-f94f-47df-a048-51b3f0dbfea8 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/3ae35b95-076f-4240-9c8f-7d63b8db9b31 HTTP/1.1" status: 204 len: 168 time: 0.2533143 May 20 04:26:32.262780 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 39280) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:32.401991 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d3eb5a9e-b2ba-43b6-8f12-9821393d3bbc admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:32.428392 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-d3eb5a9e-b2ba-43b6-8f12-9821393d3bbc admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:32.434224 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-d3eb5a9e-b2ba-43b6-8f12-9821393d3bbc admin admin] Usage tracker for resource:security_group_rule and tenant:0884374ccaef4dc3ac6582b8a011871a is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:32.438173 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-d3eb5a9e-b2ba-43b6-8f12-9821393d3bbc admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:32.444938 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-d3eb5a9e-b2ba-43b6-8f12-9821393d3bbc admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:32.597632 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d3eb5a9e-b2ba-43b6-8f12-9821393d3bbc admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:32.608993 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d3eb5a9e-b2ba-43b6-8f12-9821393d3bbc admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:32.621592 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-3835b7df-d700-4e40-817f-5e345634cc24 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:32.688593 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d3eb5a9e-b2ba-43b6-8f12-9821393d3bbc admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=83a4c430-9e50-4c79-9d20-bbb61572731b,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:32.695442 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-d3eb5a9e-b2ba-43b6-8f12-9821393d3bbc admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=0884374ccaef4dc3ac6582b8a011871a&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4291432 May 20 04:26:32.696556 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 39302) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:32.707298 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-d3eb5a9e-b2ba-43b6-8f12-9821393d3bbc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:32.807833 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a0344952-c97a-4fd2-ae5a-4d32744d171f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:32.852787 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a0344952-c97a-4fd2-ae5a-4d32744d171f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:32.889794 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6f7fa1be-f4b4-4c32-a051-35ea5dfeab90 req-0af0c2fb-131a-4b7b-b406-9255016bd5b1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:32.904975 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6f7fa1be-f4b4-4c32-a051-35ea5dfeab90 req-0af0c2fb-131a-4b7b-b406-9255016bd5b1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1339829 May 20 04:26:32.956482 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a0344952-c97a-4fd2-ae5a-4d32744d171f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:32.959508 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6f7fa1be-f4b4-4c32-a051-35ea5dfeab90 req-a8f477db-a21e-4dd6-ae7a-78b06b73a55c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0536499 May 20 04:26:32.964022 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-02bbc84d-579e-478a-993e-03c846553b9b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:32.993068 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a0344952-c97a-4fd2-ae5a-4d32744d171f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=83a4c430-9e50-4c79-9d20-bbb61572731b,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:32.996154 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-a0344952-c97a-4fd2-ae5a-4d32744d171f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.032s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:32.997338 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a0344952-c97a-4fd2-ae5a-4d32744d171f admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:32.999651 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-a0344952-c97a-4fd2-ae5a-4d32744d171f admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/83a4c430-9e50-4c79-9d20-bbb61572731b HTTP/1.1" status: 204 len: 168 time: 0.3003960 May 20 04:26:33.439776 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 39316) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:33.464005 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-bb717663-9293-41ed-b5ec-3d358fca98da admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:33.480242 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-bb717663-9293-41ed-b5ec-3d358fca98da admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:33.484365 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-bb717663-9293-41ed-b5ec-3d358fca98da admin admin] Usage tracker for resource:security_group_rule and tenant:54b96262146147d1a4ebea4810f05e2c is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:33.487590 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-bb717663-9293-41ed-b5ec-3d358fca98da admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:33.493364 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-bb717663-9293-41ed-b5ec-3d358fca98da admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:33.588278 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-bb717663-9293-41ed-b5ec-3d358fca98da admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:33.594803 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-bb717663-9293-41ed-b5ec-3d358fca98da admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:33.603763 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-82233cf5-3cb5-4b23-882a-43c4eba5b179 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:33.647551 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-bb717663-9293-41ed-b5ec-3d358fca98da admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=54b96262146147d1a4ebea4810f05e2c&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2048624 May 20 04:26:33.648990 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 39324) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:33.649506 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bb717663-9293-41ed-b5ec-3d358fca98da admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=dff56088-3834-44d4-b0ab-9a5dcdc03334,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:33.652784 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-bb717663-9293-41ed-b5ec-3d358fca98da admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:33.721500 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-af6cb013-38ca-4746-96ff-9b3a65048967 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:33.791320 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-af6cb013-38ca-4746-96ff-9b3a65048967 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:33.862395 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-af6cb013-38ca-4746-96ff-9b3a65048967 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:33.870932 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-75f8ac58-3baf-40b4-bbd5-d0a729a9374c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:33.900366 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-af6cb013-38ca-4746-96ff-9b3a65048967 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=dff56088-3834-44d4-b0ab-9a5dcdc03334,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:33.903111 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-af6cb013-38ca-4746-96ff-9b3a65048967 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.033s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:33.922801 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-af6cb013-38ca-4746-96ff-9b3a65048967 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:33.926676 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-af6cb013-38ca-4746-96ff-9b3a65048967 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/dff56088-3834-44d4-b0ab-9a5dcdc03334 HTTP/1.1" status: 204 len: 168 time: 0.2740421 May 20 04:26:34.186668 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-30cf17eb-c9b8-4254-befa-d88e3c2b3610 req-cd990cbd-24f4-4f69-b789-ceada295770f tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:34.188399 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-30cf17eb-c9b8-4254-befa-d88e3c2b3610 req-cd990cbd-24f4-4f69-b789-ceada295770f tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.0911176 May 20 04:26:34.253868 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-30cf17eb-c9b8-4254-befa-d88e3c2b3610 req-e285cbe2-9546-4554-b970-e57ff6520cd0 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0513041 May 20 04:26:35.575293 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2b01dab8-b66e-4322-8227-9496cf63eb38 req-bfc4c8a2-5b0d-4bb9-8e76-84ecbfa7d361 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:35.575293 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b01dab8-b66e-4322-8227-9496cf63eb38 req-bfc4c8a2-5b0d-4bb9-8e76-84ecbfa7d361 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1881108 May 20 04:26:35.605158 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:35.659388 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b01dab8-b66e-4322-8227-9496cf63eb38 req-9ba23635-4739-41a2-860e-fc48587055c7 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0567727 May 20 04:26:36.873025 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-27acabf3-3473-49a8-87b8-19d1a266c2fc req-a43f1add-8195-4262-ada8-027194430c36 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:36.874598 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-27acabf3-3473-49a8-87b8-19d1a266c2fc req-a43f1add-8195-4262-ada8-027194430c36 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1105375 May 20 04:26:36.958922 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-27acabf3-3473-49a8-87b8-19d1a266c2fc req-ecf14f18-9114-482c-8be4-9c5455225401 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0765719 May 20 04:26:37.676309 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 39472) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:37.815303 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-85330b5d-dbbf-46bf-a225-d272df7a5443 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:37.829454 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-85330b5d-dbbf-46bf-a225-d272df7a5443 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:37.833324 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-85330b5d-dbbf-46bf-a225-d272df7a5443 admin admin] Usage tracker for resource:security_group_rule and tenant:1f35e821a5544f65ae57638583587889 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:37.836065 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-85330b5d-dbbf-46bf-a225-d272df7a5443 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:37.842254 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-85330b5d-dbbf-46bf-a225-d272df7a5443 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:37.925502 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-85330b5d-dbbf-46bf-a225-d272df7a5443 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:37.930062 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-85330b5d-dbbf-46bf-a225-d272df7a5443 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:37.942266 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d6f98c60-a416-40d9-b9a6-8b7f7c7d2cff admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:37.981598 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-85330b5d-dbbf-46bf-a225-d272df7a5443 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=1f35e821a5544f65ae57638583587889&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3042088 May 20 04:26:37.984595 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-85330b5d-dbbf-46bf-a225-d272df7a5443 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=2d18a4fb-8613-48ea-a34b-c7190dcb3915,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:37.985608 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 39478) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:37.988335 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-85330b5d-dbbf-46bf-a225-d272df7a5443 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:38.048567 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8fb781eb-47d1-4b01-be8f-923542dad43f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:38.074581 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8fb781eb-47d1-4b01-be8f-923542dad43f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:38.135951 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8fb781eb-47d1-4b01-be8f-923542dad43f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:38.141508 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:38.154680 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8fb781eb-47d1-4b01-be8f-923542dad43f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=2d18a4fb-8613-48ea-a34b-c7190dcb3915,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:38.159132 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-8fb781eb-47d1-4b01-be8f-923542dad43f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:38.161247 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dfda3dcb-b3de-4c26-a162-2c637d5326f2 req-2fca18a0-9a32-4106-93ef-d8c135860b77 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:38.164570 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-dfda3dcb-b3de-4c26-a162-2c637d5326f2 req-2fca18a0-9a32-4106-93ef-d8c135860b77 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.0877645 May 20 04:26:38.165562 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8fb781eb-47d1-4b01-be8f-923542dad43f admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:38.166711 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-8fb781eb-47d1-4b01-be8f-923542dad43f admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/2d18a4fb-8613-48ea-a34b-c7190dcb3915 HTTP/1.1" status: 204 len: 168 time: 0.1800203 May 20 04:26:38.244531 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-dfda3dcb-b3de-4c26-a162-2c637d5326f2 req-de0b6ed2-4160-4c97-91cb-d8e1c9262a83 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0683596 May 20 04:26:38.519525 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 39490) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:38.538095 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-40e862ce-9dd8-4cbe-a4a9-f10da47d5364 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:38.552819 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-40e862ce-9dd8-4cbe-a4a9-f10da47d5364 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:38.556975 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-40e862ce-9dd8-4cbe-a4a9-f10da47d5364 admin admin] Usage tracker for resource:security_group_rule and tenant:3ff0b29639634e3aa56abacf98b1a114 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:38.559872 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-40e862ce-9dd8-4cbe-a4a9-f10da47d5364 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:38.566193 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-40e862ce-9dd8-4cbe-a4a9-f10da47d5364 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:38.667859 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-40e862ce-9dd8-4cbe-a4a9-f10da47d5364 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:38.675022 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-40e862ce-9dd8-4cbe-a4a9-f10da47d5364 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:38.695227 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-be054bc4-7ef1-41e0-8770-4ee91ef821d6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:38.717394 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 39498) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:38.718007 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-40e862ce-9dd8-4cbe-a4a9-f10da47d5364 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=3ff0b29639634e3aa56abacf98b1a114&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1967661 May 20 04:26:38.737327 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-40e862ce-9dd8-4cbe-a4a9-f10da47d5364 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=2a78ba6a-dbaa-494a-9221-9769944f2925,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:38.750439 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-40e862ce-9dd8-4cbe-a4a9-f10da47d5364 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:38.810408 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8961276a-5476-4314-9288-372a2eaaf8cc admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:38.846749 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8961276a-5476-4314-9288-372a2eaaf8cc admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:38.916579 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8961276a-5476-4314-9288-372a2eaaf8cc admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:38.922170 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-70a426a0-eac4-4b93-a3ba-3d58a1ee6187 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:38.934294 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8961276a-5476-4314-9288-372a2eaaf8cc admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=2a78ba6a-dbaa-494a-9221-9769944f2925,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:38.937331 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-8961276a-5476-4314-9288-372a2eaaf8cc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:38.939538 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8961276a-5476-4314-9288-372a2eaaf8cc admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:38.940873 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-8961276a-5476-4314-9288-372a2eaaf8cc admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/2a78ba6a-dbaa-494a-9221-9769944f2925 HTTP/1.1" status: 204 len: 168 time: 0.2213335 May 20 04:26:39.110409 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 39514) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:39.264156 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-67526a2a-c7da-4fb8-8e4b-feca7051d709 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:39.282693 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-67526a2a-c7da-4fb8-8e4b-feca7051d709 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:39.287131 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-67526a2a-c7da-4fb8-8e4b-feca7051d709 admin admin] Usage tracker for resource:security_group_rule and tenant:5b5a77e99b304e2ab73bc1c9b219c84a is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:39.290380 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-67526a2a-c7da-4fb8-8e4b-feca7051d709 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:39.298919 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-67526a2a-c7da-4fb8-8e4b-feca7051d709 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:39.392915 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-67526a2a-c7da-4fb8-8e4b-feca7051d709 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:39.411914 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-67526a2a-c7da-4fb8-8e4b-feca7051d709 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:39.418862 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-d3eb5a9e-b2ba-43b6-8f12-9821393d3bbc admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:39.463789 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-67526a2a-c7da-4fb8-8e4b-feca7051d709 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=5b5a77e99b304e2ab73bc1c9b219c84a&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3518348 May 20 04:26:39.465261 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 39530) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:39.468813 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-67526a2a-c7da-4fb8-8e4b-feca7051d709 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=43f3dd75-2536-4975-9035-ba8ea56ab64d,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:39.471953 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-67526a2a-c7da-4fb8-8e4b-feca7051d709 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:39.552947 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-f9d43527-602c-458f-b156-bb4321cba569 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:39.598362 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-f9d43527-602c-458f-b156-bb4321cba569 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:39.632604 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1492a51a-d60a-41e3-8966-73e8b6adacae req-f917f461-4174-45f1-8767-2263d05705dc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:39.635075 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1492a51a-d60a-41e3-8966-73e8b6adacae req-f917f461-4174-45f1-8767-2263d05705dc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1118264 May 20 04:26:39.697411 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-f9d43527-602c-458f-b156-bb4321cba569 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:39.702752 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-25c8ac88-97c6-490a-ae61-a431df424432 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:39.708528 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1492a51a-d60a-41e3-8966-73e8b6adacae req-bdb8c2de-e319-4e5c-ade4-2f35240cd508 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0650613 May 20 04:26:39.716301 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f9d43527-602c-458f-b156-bb4321cba569 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=43f3dd75-2536-4975-9035-ba8ea56ab64d,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:39.719865 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-f9d43527-602c-458f-b156-bb4321cba569 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:39.723136 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-f9d43527-602c-458f-b156-bb4321cba569 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:39.727744 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-f9d43527-602c-458f-b156-bb4321cba569 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/43f3dd75-2536-4975-9035-ba8ea56ab64d HTTP/1.1" status: 204 len: 168 time: 0.2613468 May 20 04:26:40.067596 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 39552) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:40.092890 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-03f1abe1-7037-48c2-bb2b-2e5ad74bfa21 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:40.108367 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-03f1abe1-7037-48c2-bb2b-2e5ad74bfa21 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:40.112905 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-03f1abe1-7037-48c2-bb2b-2e5ad74bfa21 admin admin] Usage tracker for resource:security_group_rule and tenant:eb9e2df0645c4abd83a063350b63f10f is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:40.116183 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-03f1abe1-7037-48c2-bb2b-2e5ad74bfa21 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:40.122286 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-03f1abe1-7037-48c2-bb2b-2e5ad74bfa21 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:40.224283 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-03f1abe1-7037-48c2-bb2b-2e5ad74bfa21 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:40.229627 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-03f1abe1-7037-48c2-bb2b-2e5ad74bfa21 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:40.238824 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-c0748f8c-ee43-4533-b6ef-4a70716d53eb admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:40.280581 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-03f1abe1-7037-48c2-bb2b-2e5ad74bfa21 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=eb9e2df0645c4abd83a063350b63f10f&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2116675 May 20 04:26:40.281495 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-03f1abe1-7037-48c2-bb2b-2e5ad74bfa21 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=4f349b21-400f-4cb6-ab9e-9ebac931fa22,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:40.283996 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 39558) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:40.298884 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-03f1abe1-7037-48c2-bb2b-2e5ad74bfa21 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.060s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:40.346537 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6aedee67-74fc-4888-84cb-e3b6ca159d48 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:40.375746 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6aedee67-74fc-4888-84cb-e3b6ca159d48 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:40.438254 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6aedee67-74fc-4888-84cb-e3b6ca159d48 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:40.445512 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:40.464678 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6aedee67-74fc-4888-84cb-e3b6ca159d48 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=4f349b21-400f-4cb6-ab9e-9ebac931fa22,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:40.467318 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-6aedee67-74fc-4888-84cb-e3b6ca159d48 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.022s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:40.482413 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6aedee67-74fc-4888-84cb-e3b6ca159d48 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:40.486882 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-6aedee67-74fc-4888-84cb-e3b6ca159d48 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/4f349b21-400f-4cb6-ab9e-9ebac931fa22 HTTP/1.1" status: 204 len: 168 time: 0.2016170 May 20 04:26:40.912548 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 39584) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:40.934899 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-f273c300-12fb-45d9-bf74-491c3aff2e48 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:40.950252 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-f273c300-12fb-45d9-bf74-491c3aff2e48 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:40.954434 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-f273c300-12fb-45d9-bf74-491c3aff2e48 admin admin] Usage tracker for resource:security_group_rule and tenant:8da1638b92704a3cb86fff25bb6fa02c is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:40.957788 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-f273c300-12fb-45d9-bf74-491c3aff2e48 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:40.965177 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-f273c300-12fb-45d9-bf74-491c3aff2e48 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:41.039118 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2ca5b844-6664-4330-84e1-8427ca7a390e req-0c065741-603a-4b6a-aee3-333783623b47 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:41.040808 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2ca5b844-6664-4330-84e1-8427ca7a390e req-0c065741-603a-4b6a-aee3-333783623b47 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1328297 May 20 04:26:41.070797 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-f273c300-12fb-45d9-bf74-491c3aff2e48 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:41.083375 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-f273c300-12fb-45d9-bf74-491c3aff2e48 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:41.101453 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-8fb781eb-47d1-4b01-be8f-923542dad43f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:41.118612 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2ca5b844-6664-4330-84e1-8427ca7a390e req-15e6b58a-5e97-4bb3-83ea-642d71349f1e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0711112 May 20 04:26:41.168074 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-f273c300-12fb-45d9-bf74-491c3aff2e48 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=8da1638b92704a3cb86fff25bb6fa02c&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2543836 May 20 04:26:41.170099 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f273c300-12fb-45d9-bf74-491c3aff2e48 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=a688e002-332d-414b-928b-9823bff80d0b,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:41.170787 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 39592) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:41.173202 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-f273c300-12fb-45d9-bf74-491c3aff2e48 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.072s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:41.238546 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bb5ed9e6-f456-4872-83ec-710f791ea7ce admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:41.266071 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bb5ed9e6-f456-4872-83ec-710f791ea7ce admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:41.330145 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bb5ed9e6-f456-4872-83ec-710f791ea7ce admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:41.336811 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9fb3fca5-141d-4af5-82e9-55f49eeb933c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:41.348620 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bb5ed9e6-f456-4872-83ec-710f791ea7ce admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=a688e002-332d-414b-928b-9823bff80d0b,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:41.360630 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-bb5ed9e6-f456-4872-83ec-710f791ea7ce admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:41.370672 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bb5ed9e6-f456-4872-83ec-710f791ea7ce admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:41.384505 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-bb5ed9e6-f456-4872-83ec-710f791ea7ce admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/a688e002-332d-414b-928b-9823bff80d0b HTTP/1.1" status: 204 len: 168 time: 0.2125700 May 20 04:26:42.565566 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3fa6b289-55d1-4a19-a74b-343f0cabdf54 req-b07a6fe0-52e5-49cb-a0c5-34d81103f974 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:42.568866 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3fa6b289-55d1-4a19-a74b-343f0cabdf54 req-b07a6fe0-52e5-49cb-a0c5-34d81103f974 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.3247116 May 20 04:26:42.609094 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-25939e05-1d08-4e12-a8fa-8b75532ccc11 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:42.620674 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3fa6b289-55d1-4a19-a74b-343f0cabdf54 req-f1481cb1-7fa3-4166-a5ab-988f871c112b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0462863 May 20 04:26:44.023294 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-abb34d44-5a5e-41e0-a2c7-05f8dfa28da6 req-dda8de64-843b-4e9f-ac29-5f5d6f19c319 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:44.024986 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-abb34d44-5a5e-41e0-a2c7-05f8dfa28da6 req-dda8de64-843b-4e9f-ac29-5f5d6f19c319 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.2380979 May 20 04:26:44.071639 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-abb34d44-5a5e-41e0-a2c7-05f8dfa28da6 req-3774c46e-ba3a-40c0-9d7a-4b5b179ea7a1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0403347 May 20 04:26:44.968116 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:45.094051 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-80be0e8d-bbe6-4fa2-8967-83815f1967d3 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:45.317658 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f6ff840b-4688-45e0-9a37-6bcd3349e26a req-1fb6b8db-b66c-45a9-b0ea-ae122b752c4c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:45.319420 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f6ff840b-4688-45e0-9a37-6bcd3349e26a req-1fb6b8db-b66c-45a9-b0ea-ae122b752c4c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.0902686 May 20 04:26:45.372201 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f6ff840b-4688-45e0-9a37-6bcd3349e26a req-28d52a97-7c0f-431f-83b4-aa069a576ea3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0461895 May 20 04:26:45.567336 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0725a580-1bf8-497d-8ecf-d649653ce281 req-6725acab-3778-4c7f-9e23-25d07c7b4951 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:45.568788 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0725a580-1bf8-497d-8ecf-d649653ce281 req-6725acab-3778-4c7f-9e23-25d07c7b4951 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=9fafdf3a29024ac588e5b24719735677&device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 1098 time: 0.0732138 May 20 04:26:45.705075 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0725a580-1bf8-497d-8ecf-d649653ce281 req-aa4a3415-4364-4e11-bdc8-f94d793f2784 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:45.706515 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0725a580-1bf8-497d-8ecf-d649653ce281 req-aa4a3415-4364-4e11-bdc8-f94d793f2784 service neutron] 10.222.0.74 "GET /v2.0/networks?id=64bcea8a-6d35-406b-a823-fa1185008174 HTTP/1.1" status: 200 len: 918 time: 0.1308076 May 20 04:26:45.810327 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0725a580-1bf8-497d-8ecf-d649653ce281 req-b385b82f-4d08-45c6-bb9a-ad067d2a6e47 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=535d5637-ae66-43cb-bd16-3543bd3b7a8b HTTP/1.1" status: 200 len: 1001 time: 0.0969946 May 20 04:26:45.872928 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0725a580-1bf8-497d-8ecf-d649653ce281 req-ce741294-fd5c-4dd7-b5fc-370bc622b5d0 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:45.874287 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0725a580-1bf8-497d-8ecf-d649653ce281 req-ce741294-fd5c-4dd7-b5fc-370bc622b5d0 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=203a3962-2282-4f51-ac92-c9695b5c0cfe HTTP/1.1" status: 200 len: 856 time: 0.0560727 May 20 04:26:45.953368 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0725a580-1bf8-497d-8ecf-d649653ce281 req-93ea5ccf-452d-482d-ae12-c76b44b5bd44 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:45.954751 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0725a580-1bf8-497d-8ecf-d649653ce281 req-93ea5ccf-452d-482d-ae12-c76b44b5bd44 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=64bcea8a-6d35-406b-a823-fa1185008174&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0731800 May 20 04:26:46.087186 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0725a580-1bf8-497d-8ecf-d649653ce281 req-9194c737-02b4-4eac-9ffb-2b40365efa63 service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1251976 May 20 04:26:46.181355 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:46.217045 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0725a580-1bf8-497d-8ecf-d649653ce281 req-779a2ec7-57d4-4d5d-bd9a-ba271847a34f service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1219406 May 20 04:26:46.592658 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a53b2a31-2cdd-432c-aaab-02f48898eae7 req-6c3a3cda-d6d1-4363-a008-f1763be415be tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:46.595541 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a53b2a31-2cdd-432c-aaab-02f48898eae7 req-6c3a3cda-d6d1-4363-a008-f1763be415be tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1105070 May 20 04:26:46.644177 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a53b2a31-2cdd-432c-aaab-02f48898eae7 req-b0b5e4c1-bfe0-42e4-98cc-000a5fd6c742 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0428050 May 20 04:26:46.671156 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 39728) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:46.784900 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-19e16154-7f87-49b5-98b5-55a35496478e admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:46.799517 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-19e16154-7f87-49b5-98b5-55a35496478e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:46.806638 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-19e16154-7f87-49b5-98b5-55a35496478e admin admin] Usage tracker for resource:security_group_rule and tenant:a197828b22c2441f8f832faa30697f9c is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:46.809711 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-19e16154-7f87-49b5-98b5-55a35496478e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:46.815891 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-19e16154-7f87-49b5-98b5-55a35496478e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:46.910269 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 39736) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:46.915905 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-19e16154-7f87-49b5-98b5-55a35496478e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:46.921106 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-19e16154-7f87-49b5-98b5-55a35496478e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:46.949760 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-36da6ca3-a044-4e91-a4f8-c37b0b0d5c57 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:46.960516 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-19e16154-7f87-49b5-98b5-55a35496478e admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=a197828b22c2441f8f832faa30697f9c&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2957335 May 20 04:26:46.963246 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 39742) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:46.980315 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-19e16154-7f87-49b5-98b5-55a35496478e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=1f5bd6bd-2498-4181-9e59-9ddf7942af5b,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:46.984093 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-19e16154-7f87-49b5-98b5-55a35496478e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.034s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:47.029340 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-be5b9fb8-64a6-4d84-9f1d-e778609f601e admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:47.065271 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e3916fb5-11b7-4a76-be10-cfa0908f1ff2 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:47.079851 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-be5b9fb8-64a6-4d84-9f1d-e778609f601e admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:47.092336 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-e3916fb5-11b7-4a76-be10-cfa0908f1ff2 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:47.102537 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-e3916fb5-11b7-4a76-be10-cfa0908f1ff2 admin admin] Usage tracker for resource:security_group_rule and tenant:279762bb6e014db89ae5186a1f1293d0 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:47.107487 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-e3916fb5-11b7-4a76-be10-cfa0908f1ff2 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:47.121378 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-e3916fb5-11b7-4a76-be10-cfa0908f1ff2 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:47.168377 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-be5b9fb8-64a6-4d84-9f1d-e778609f601e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:47.174377 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-602010bf-d62e-4e24-b090-392bcbe40f2e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:47.192041 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-be5b9fb8-64a6-4d84-9f1d-e778609f601e admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=1f5bd6bd-2498-4181-9e59-9ddf7942af5b,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:47.197390 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-be5b9fb8-64a6-4d84-9f1d-e778609f601e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:47.232124 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-be5b9fb8-64a6-4d84-9f1d-e778609f601e admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:47.233208 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-be5b9fb8-64a6-4d84-9f1d-e778609f601e admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/1f5bd6bd-2498-4181-9e59-9ddf7942af5b HTTP/1.1" status: 204 len: 168 time: 0.2681289 May 20 04:26:47.246925 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e3916fb5-11b7-4a76-be10-cfa0908f1ff2 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:47.252052 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e3916fb5-11b7-4a76-be10-cfa0908f1ff2 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:47.258370 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-f273c300-12fb-45d9-bf74-491c3aff2e48 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:47.297455 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e3916fb5-11b7-4a76-be10-cfa0908f1ff2 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=b92e97eb-5555-4196-a1e4-c5c0ec18b65e,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:47.304492 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-e3916fb5-11b7-4a76-be10-cfa0908f1ff2 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=279762bb6e014db89ae5186a1f1293d0&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3931584 May 20 04:26:47.304705 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 39748) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:47.305558 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e3916fb5-11b7-4a76-be10-cfa0908f1ff2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:47.368966 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d7fc5fcf-d139-4e50-b5cd-edec41470990 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:47.394294 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d7fc5fcf-d139-4e50-b5cd-edec41470990 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:47.449382 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d7fc5fcf-d139-4e50-b5cd-edec41470990 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:47.454571 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-68ac8851-9d6c-4364-875d-1b8c3e6cbc0e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:47.468332 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d7fc5fcf-d139-4e50-b5cd-edec41470990 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=b92e97eb-5555-4196-a1e4-c5c0ec18b65e,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:47.472413 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-d7fc5fcf-d139-4e50-b5cd-edec41470990 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:47.478769 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d7fc5fcf-d139-4e50-b5cd-edec41470990 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:47.479895 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-d7fc5fcf-d139-4e50-b5cd-edec41470990 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/b92e97eb-5555-4196-a1e4-c5c0ec18b65e HTTP/1.1" status: 204 len: 168 time: 0.1739590 May 20 04:26:47.524618 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 39756) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:47.541952 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-76990c91-557a-40cb-984d-21e679e1bb19 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:47.556643 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-76990c91-557a-40cb-984d-21e679e1bb19 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:47.560378 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-76990c91-557a-40cb-984d-21e679e1bb19 admin admin] Usage tracker for resource:security_group_rule and tenant:111a6b7cd2644b9ca1e1796a2ae938d5 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:47.563088 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-76990c91-557a-40cb-984d-21e679e1bb19 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:47.569072 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-76990c91-557a-40cb-984d-21e679e1bb19 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:47.655526 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-76990c91-557a-40cb-984d-21e679e1bb19 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:47.661373 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-76990c91-557a-40cb-984d-21e679e1bb19 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:47.671188 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-af6cb013-38ca-4746-96ff-9b3a65048967 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:47.714326 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-76990c91-557a-40cb-984d-21e679e1bb19 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=111a6b7cd2644b9ca1e1796a2ae938d5&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1884832 May 20 04:26:47.715323 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-76990c91-557a-40cb-984d-21e679e1bb19 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=3b006dcd-e0d3-4f3b-a487-5bd641b7f450,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:47.717630 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 39766) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:47.718785 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-76990c91-557a-40cb-984d-21e679e1bb19 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:47.786263 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-36dfed38-036e-419d-b5a6-8b4ce4bab3e8 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:47.814209 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-36dfed38-036e-419d-b5a6-8b4ce4bab3e8 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:47.823232 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:47.861615 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 39770) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:47.878625 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a3769a15-b281-4c70-a7e3-bf3c02777a2d admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:47.887874 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-809e078d-0545-4d3f-a712-9cb2553fbf4c req-6593204e-4670-47db-9e12-1b5f0d2793e0 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:47.889332 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-809e078d-0545-4d3f-a712-9cb2553fbf4c req-6593204e-4670-47db-9e12-1b5f0d2793e0 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.0924060 May 20 04:26:47.897520 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-a3769a15-b281-4c70-a7e3-bf3c02777a2d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:47.901530 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-36dfed38-036e-419d-b5a6-8b4ce4bab3e8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:47.905126 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-a3769a15-b281-4c70-a7e3-bf3c02777a2d admin admin] Usage tracker for resource:security_group_rule and tenant:4e2c8eca501247b7853efece072ce1c0 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:47.907675 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-1843ae6f-1d1f-49e0-bf59-aa2560cc247d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:47.920046 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-a3769a15-b281-4c70-a7e3-bf3c02777a2d admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:47.927239 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-36dfed38-036e-419d-b5a6-8b4ce4bab3e8 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=3b006dcd-e0d3-4f3b-a487-5bd641b7f450,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:47.929602 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-36dfed38-036e-419d-b5a6-8b4ce4bab3e8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.022s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:47.930686 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-a3769a15-b281-4c70-a7e3-bf3c02777a2d admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:47.935034 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-809e078d-0545-4d3f-a712-9cb2553fbf4c req-d37ea009-1b13-4a76-950e-19ee65865f89 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0396883 May 20 04:26:47.951022 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-36dfed38-036e-419d-b5a6-8b4ce4bab3e8 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:47.956150 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-36dfed38-036e-419d-b5a6-8b4ce4bab3e8 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/3b006dcd-e0d3-4f3b-a487-5bd641b7f450 HTTP/1.1" status: 204 len: 168 time: 0.2363281 May 20 04:26:48.027861 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a3769a15-b281-4c70-a7e3-bf3c02777a2d admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:48.032117 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a3769a15-b281-4c70-a7e3-bf3c02777a2d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:48.039971 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-be5b9fb8-64a6-4d84-9f1d-e778609f601e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:48.077851 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-a3769a15-b281-4c70-a7e3-bf3c02777a2d admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=4e2c8eca501247b7853efece072ce1c0&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2151988 May 20 04:26:48.080882 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 39776) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:48.083521 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a3769a15-b281-4c70-a7e3-bf3c02777a2d admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=60f0ac3b-f2b7-4443-b7d4-d9afb80b5612,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:48.097388 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-a3769a15-b281-4c70-a7e3-bf3c02777a2d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.057s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:48.145366 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-32b2ea62-71b5-42c4-a8f0-5f343b00d2c8 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:48.171171 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-32b2ea62-71b5-42c4-a8f0-5f343b00d2c8 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:48.223171 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-32b2ea62-71b5-42c4-a8f0-5f343b00d2c8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:48.228148 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-8961276a-5476-4314-9288-372a2eaaf8cc admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:48.240588 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-32b2ea62-71b5-42c4-a8f0-5f343b00d2c8 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=60f0ac3b-f2b7-4443-b7d4-d9afb80b5612,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:48.242872 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-32b2ea62-71b5-42c4-a8f0-5f343b00d2c8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:48.246262 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-32b2ea62-71b5-42c4-a8f0-5f343b00d2c8 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:48.247401 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-32b2ea62-71b5-42c4-a8f0-5f343b00d2c8 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/60f0ac3b-f2b7-4443-b7d4-d9afb80b5612 HTTP/1.1" status: 204 len: 168 time: 0.1655931 May 20 04:26:49.141619 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 535d5637-ae66-43cb-bd16-3543bd3b7a8b no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:26:49.276839 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:49.312561 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'c2ca950f-78ff-4e44-9231-e8e19f4aee5d', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': '535d5637-ae66-43cb-bd16-3543bd3b7a8b'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:26:49.325369 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 535d5637-ae66-43cb-bd16-3543bd3b7a8b, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=535d5637-ae66-43cb-bd16-3543bd3b7a8b,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:26:49.330323 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:49.336168 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-28d40cc2-394d-467f-8c50-20a78bcf5f02 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:49.370354 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0b5149c-e867-450e-8212-1b1dac4507fb req-0d39d72d-3350-4b9c-af7a-526c8b865d16 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:49.378149 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d0b5149c-e867-450e-8212-1b1dac4507fb req-0d39d72d-3350-4b9c-af7a-526c8b865d16 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 945 time: 0.1017230 May 20 04:26:49.402315 dv-cinder-782405-9-iscsi neutron-server[91645]: WARNING neutron.notifiers.nova [-] Nova event: {'server_uuid': 'c2ca950f-78ff-4e44-9231-e8e19f4aee5d', 'name': 'network-vif-unplugged', 'status': 'failed', 'tag': '535d5637-ae66-43cb-bd16-3543bd3b7a8b', 'code': 422} returned with failed status May 20 04:26:49.438570 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=535d5637-ae66-43cb-bd16-3543bd3b7a8b,revision_number=8']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:49.445643 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d0b5149c-e867-450e-8212-1b1dac4507fb req-18631d27-edc5-4577-a3ea-0fe872d1087c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0665412 May 20 04:26:49.450059 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:50.643706 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3a54bc69-9f55-471e-bb03-47f44471e1ab req-4dea72c0-c98f-40b1-9c9e-791bdc1ff1e3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:50.645184 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3a54bc69-9f55-471e-bb03-47f44471e1ab req-4dea72c0-c98f-40b1-9c9e-791bdc1ff1e3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.0827017 May 20 04:26:50.693674 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3a54bc69-9f55-471e-bb03-47f44471e1ab req-6e5cd821-7c02-4f3e-bcbf-97c038094200 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0421474 May 20 04:26:50.907241 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 39912) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:51.130295 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9b79ab25-2bfc-4551-8cb9-6d73244c86b2 req-ba8bb90a-9735-44db-9545-e130d40a439e service neutron] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d&fields=id&fields=resource_request HTTP/1.1" status: 200 len: 255 time: 0.2218199 May 20 04:26:51.909839 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d5b1c1f3-e749-4b55-8ecc-e24518b6edaf req-a01a1133-5268-48ac-ba20-73a9d1ab2d80 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:51.912155 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d5b1c1f3-e749-4b55-8ecc-e24518b6edaf req-a01a1133-5268-48ac-ba20-73a9d1ab2d80 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.0900638 May 20 04:26:51.958503 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d5b1c1f3-e749-4b55-8ecc-e24518b6edaf req-05d5d750-0c59-43b6-b9ad-6bc53edab9a3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0401568 May 20 04:26:53.146101 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9b79ab25-2bfc-4551-8cb9-6d73244c86b2 req-8bac1e62-e2e1-4523-af87-2879ce0426b1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:53.148066 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9b79ab25-2bfc-4551-8cb9-6d73244c86b2 req-8bac1e62-e2e1-4523-af87-2879ce0426b1 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d&tenant_id=9fafdf3a29024ac588e5b24719735677 HTTP/1.1" status: 200 len: 1096 time: 0.1021862 May 20 04:26:53.155480 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 535d5637-ae66-43cb-bd16-3543bd3b7a8b no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:26:53.210131 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 39976) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:53.303500 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9b79ab25-2bfc-4551-8cb9-6d73244c86b2 req-94b36c4a-93a2-48f4-99b4-5624476a449e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:53.308256 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9b79ab25-2bfc-4551-8cb9-6d73244c86b2 req-94b36c4a-93a2-48f4-99b4-5624476a449e service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=9fafdf3a29024ac588e5b24719735677&device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 1096 time: 0.1344233 May 20 04:26:53.407084 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-92549aa6-8527-4538-acee-14bb37dfc870 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:53.423078 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-92549aa6-8527-4538-acee-14bb37dfc870 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:53.427028 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-92549aa6-8527-4538-acee-14bb37dfc870 admin admin] Usage tracker for resource:security_group_rule and tenant:3efe472f226a422d92819bb7e431f94b is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:53.430043 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-92549aa6-8527-4538-acee-14bb37dfc870 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:53.436307 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-92549aa6-8527-4538-acee-14bb37dfc870 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:53.521813 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-92549aa6-8527-4538-acee-14bb37dfc870 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:53.526701 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8d3494c4-e182-4402-af5a-ec1edc96d8fc req-153a039c-c0d9-4e8c-a201-375cf3c27b5a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:53.528685 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8d3494c4-e182-4402-af5a-ec1edc96d8fc req-153a039c-c0d9-4e8c-a201-375cf3c27b5a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.2357678 May 20 04:26:53.530929 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9b79ab25-2bfc-4551-8cb9-6d73244c86b2 req-6d619ce7-956e-43ae-b783-202a77bce97b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:53.532332 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9b79ab25-2bfc-4551-8cb9-6d73244c86b2 req-6d619ce7-956e-43ae-b783-202a77bce97b service neutron] 10.222.0.74 "GET /v2.0/networks?id=64bcea8a-6d35-406b-a823-fa1185008174 HTTP/1.1" status: 200 len: 918 time: 0.1704662 May 20 04:26:53.532710 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-92549aa6-8527-4538-acee-14bb37dfc870 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:53.542159 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-51fc048c-83a8-41f0-92b3-bedf34d4be55 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:53.586656 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-92549aa6-8527-4538-acee-14bb37dfc870 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=3efe472f226a422d92819bb7e431f94b&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3754342 May 20 04:26:53.587268 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 40000) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:53.590838 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-92549aa6-8527-4538-acee-14bb37dfc870 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=3efc357c-e314-43e1-8db8-dd12845d393a,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:53.594447 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-92549aa6-8527-4538-acee-14bb37dfc870 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.052s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:53.651025 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8d3494c4-e182-4402-af5a-ec1edc96d8fc req-93b06d4f-0d05-4986-9bb7-ca45b41bff1a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1160254 May 20 04:26:53.658773 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-42d44108-3825-4080-a8ff-c08242d17cae admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:53.667422 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9b79ab25-2bfc-4551-8cb9-6d73244c86b2 req-b462ff40-d144-4940-88f0-f4906449e770 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=535d5637-ae66-43cb-bd16-3543bd3b7a8b HTTP/1.1" status: 200 len: 999 time: 0.1186950 May 20 04:26:53.686616 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-42d44108-3825-4080-a8ff-c08242d17cae admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:53.735771 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9b79ab25-2bfc-4551-8cb9-6d73244c86b2 req-44d6e6d1-8b03-4b0b-95f3-7262e704baa2 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:53.738190 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9b79ab25-2bfc-4551-8cb9-6d73244c86b2 req-44d6e6d1-8b03-4b0b-95f3-7262e704baa2 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=203a3962-2282-4f51-ac92-c9695b5c0cfe HTTP/1.1" status: 200 len: 856 time: 0.0643268 May 20 04:26:53.750168 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-42d44108-3825-4080-a8ff-c08242d17cae admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:53.754181 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-03f1abe1-7037-48c2-bb2b-2e5ad74bfa21 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:53.767340 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-42d44108-3825-4080-a8ff-c08242d17cae admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=3efc357c-e314-43e1-8db8-dd12845d393a,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:53.769627 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-42d44108-3825-4080-a8ff-c08242d17cae admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:53.773341 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-42d44108-3825-4080-a8ff-c08242d17cae admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:53.774682 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-42d44108-3825-4080-a8ff-c08242d17cae admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/3efc357c-e314-43e1-8db8-dd12845d393a HTTP/1.1" status: 204 len: 168 time: 0.1857033 May 20 04:26:53.829229 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9b79ab25-2bfc-4551-8cb9-6d73244c86b2 req-38a29408-ca7c-4d99-8da8-0f1c5bc7bcf1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:53.830732 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9b79ab25-2bfc-4551-8cb9-6d73244c86b2 req-38a29408-ca7c-4d99-8da8-0f1c5bc7bcf1 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=64bcea8a-6d35-406b-a823-fa1185008174&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0855043 May 20 04:26:53.969179 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9b79ab25-2bfc-4551-8cb9-6d73244c86b2 req-4a036d2f-ae9e-40b1-b7ec-d24d4c5bf4ae service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1309738 May 20 04:26:54.229530 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9b79ab25-2bfc-4551-8cb9-6d73244c86b2 req-29431b62-8470-4e19-84da-781cac688c74 service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2485993 May 20 04:26:54.861926 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-197c19fa-d53e-4b79-bb45-41cdbf4c7738 req-b17077ae-a1d5-4fbb-a6b7-e41857c97856 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:54.863646 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-197c19fa-d53e-4b79-bb45-41cdbf4c7738 req-b17077ae-a1d5-4fbb-a6b7-e41857c97856 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.0874290 May 20 04:26:54.933711 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-197c19fa-d53e-4b79-bb45-41cdbf4c7738 req-b966e76b-053e-44a9-a387-a8088f068286 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0632913 May 20 04:26:56.068497 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 40128) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:56.220603 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ec0da560-0b67-40fc-9e2b-149122e77bae req-3b7b3e74-5a96-495f-a801-6b18dc11d792 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:56.223683 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ec0da560-0b67-40fc-9e2b-149122e77bae req-3b7b3e74-5a96-495f-a801-6b18dc11d792 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.1585894 May 20 04:26:56.282593 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ec0da560-0b67-40fc-9e2b-149122e77bae req-253483ca-c131-4e64-b5f8-8a35b08c0369 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0528953 May 20 04:26:56.343678 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4fcc8c00-8f3c-4262-9d97-c367b7298369 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:56.358348 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-4fcc8c00-8f3c-4262-9d97-c367b7298369 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:56.362582 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-4fcc8c00-8f3c-4262-9d97-c367b7298369 admin admin] Usage tracker for resource:security_group_rule and tenant:7747c4a0dcbd45cb8af02deb6fe933dd is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:56.365490 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-4fcc8c00-8f3c-4262-9d97-c367b7298369 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:56.371673 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-4fcc8c00-8f3c-4262-9d97-c367b7298369 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:56.462549 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4fcc8c00-8f3c-4262-9d97-c367b7298369 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:56.467632 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-4fcc8c00-8f3c-4262-9d97-c367b7298369 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:56.477573 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-32b2ea62-71b5-42c4-a8f0-5f343b00d2c8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:56.519419 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-4fcc8c00-8f3c-4262-9d97-c367b7298369 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=7747c4a0dcbd45cb8af02deb6fe933dd&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4495497 May 20 04:26:56.520292 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4fcc8c00-8f3c-4262-9d97-c367b7298369 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=c3ba0491-9b79-4eb7-b671-81e33b7e1aa5,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:56.522810 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 40138) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:56.523130 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-4fcc8c00-8f3c-4262-9d97-c367b7298369 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:56.596065 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-be19a9d7-e273-4a1e-84cf-984ac9d58fc3 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:56.623777 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-be19a9d7-e273-4a1e-84cf-984ac9d58fc3 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:56.677165 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-be19a9d7-e273-4a1e-84cf-984ac9d58fc3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:56.680803 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-3377ee61-7669-4183-bff8-18f4c142cb1e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:56.692675 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-be19a9d7-e273-4a1e-84cf-984ac9d58fc3 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=c3ba0491-9b79-4eb7-b671-81e33b7e1aa5,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:56.695466 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-be19a9d7-e273-4a1e-84cf-984ac9d58fc3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:56.702196 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-be19a9d7-e273-4a1e-84cf-984ac9d58fc3 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:56.703302 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-be19a9d7-e273-4a1e-84cf-984ac9d58fc3 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/c3ba0491-9b79-4eb7-b671-81e33b7e1aa5 HTTP/1.1" status: 204 len: 168 time: 0.1794801 May 20 04:26:56.995845 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 40146) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:57.016547 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7e1a0a60-c88d-4fa0-afc4-1304c3301432 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:57.030846 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-7e1a0a60-c88d-4fa0-afc4-1304c3301432 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:57.035171 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-7e1a0a60-c88d-4fa0-afc4-1304c3301432 admin admin] Usage tracker for resource:security_group_rule and tenant:522fd84825db4229980ba1a7bf804a92 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:57.037921 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-7e1a0a60-c88d-4fa0-afc4-1304c3301432 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:57.043579 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-7e1a0a60-c88d-4fa0-afc4-1304c3301432 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:57.133132 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7e1a0a60-c88d-4fa0-afc4-1304c3301432 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:57.140703 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7e1a0a60-c88d-4fa0-afc4-1304c3301432 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:57.167376 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-8fb459cd-2b0f-49ac-b6f1-f17ac796e065 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:57.180058 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-7e1a0a60-c88d-4fa0-afc4-1304c3301432 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=522fd84825db4229980ba1a7bf804a92&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1831620 May 20 04:26:57.182656 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 40150) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:57.211758 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7e1a0a60-c88d-4fa0-afc4-1304c3301432 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=b77c9381-d893-4409-9895-1d3c1ebfc56b,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:57.221051 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-7e1a0a60-c88d-4fa0-afc4-1304c3301432 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:57.271717 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b7c033a2-8bfe-4a59-b278-426ec5f4b7c0 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:57.324983 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b7c033a2-8bfe-4a59-b278-426ec5f4b7c0 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:57.382078 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b7c033a2-8bfe-4a59-b278-426ec5f4b7c0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:57.385717 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-4bbb5526-1dc4-49c8-ac3c-a4e7602f28f4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:57.396790 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b7c033a2-8bfe-4a59-b278-426ec5f4b7c0 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=b77c9381-d893-4409-9895-1d3c1ebfc56b,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:57.400960 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-b7c033a2-8bfe-4a59-b278-426ec5f4b7c0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:57.406548 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b7c033a2-8bfe-4a59-b278-426ec5f4b7c0 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:57.407727 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-b7c033a2-8bfe-4a59-b278-426ec5f4b7c0 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/b77c9381-d893-4409-9895-1d3c1ebfc56b HTTP/1.1" status: 204 len: 168 time: 0.2240958 May 20 04:26:57.491870 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1a5d0124-76e6-4ccd-9f97-616bb0d03282 req-5d4f982f-d246-4f65-ab95-9a09f4d35a5d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:57.493334 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1a5d0124-76e6-4ccd-9f97-616bb0d03282 req-5d4f982f-d246-4f65-ab95-9a09f4d35a5d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.0812082 May 20 04:26:57.539838 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1a5d0124-76e6-4ccd-9f97-616bb0d03282 req-d90b0603-9c66-4eb3-b247-b8932aa2daa2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0407810 May 20 04:26:58.259438 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 40190) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:58.365734 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Request body: {'network': {'name': 'tempest-VolumesActionsTest-108595079-network', 'tenant_id': 'c57d56efd5fb40069159f50973655a5d'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:26:58.370371 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:58.374461 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Usage tracker for resource:network and tenant:c57d56efd5fb40069159f50973655a5d is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:58.377194 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:58.383402 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:58.393533 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Made reservation on behalf of c57d56efd5fb40069159f50973655a5d for: {'network': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:26:58.395962 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-585852'] for network, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:58.407859 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:58.427500 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:58.432227 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Usage tracker for resource:security_group_rule and tenant:c57d56efd5fb40069159f50973655a5d is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:58.435613 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:58.442588 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:58.715310 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:58.723417 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:58.731435 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-6aedee67-74fc-4888-84cb-e3b6ca159d48 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:58.774030 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 809} {{(pid=91643) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 20 04:26:58.780899 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e551ae97-21ce-4dbb-b450-b8fe6812efec,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:58.797038 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:58.797470 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:58.797910 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:58.798275 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:58.798624 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:58.798978 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:58.799355 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:58.799710 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:58.800167 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:58.800523 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:58.800872 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:58.801240 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:58.805238 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:58.806555 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:58.806901 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.segments_db [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Added segment 0d9a1ad5-c1fc-43c7-8293-6032c9456478 of type vlan for network e3248a7c-d6dc-404f-9bb7-8e87d070d1cb May 20 04:26:58.848637 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cd272830-f29a-4627-a1ca-3a9532a85dcf req-8fa17002-584d-4cd6-9be1-58a75eaaa64d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:58.850128 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cd272830-f29a-4627-a1ca-3a9532a85dcf req-8fa17002-584d-4cd6-9be1-58a75eaaa64d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.1782598 May 20 04:26:58.851933 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8760688516720'] for network, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:58.915683 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cd272830-f29a-4627-a1ca-3a9532a85dcf req-0719fdd3-e084-404a-85a9-c023fe6604df tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0571184 May 20 04:26:58.916818 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915'] for network, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:58.918142 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:58.918823 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:58.924211 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:59.015339 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:26:59.035949 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 40212) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:26:59.036611 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] 10.222.0.39 "POST /v2.0/networks HTTP/1.1" status: 201 len: 881 time: 0.7754366 May 20 04:26:59.039800 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Pushing event updated for resources: {'Network': ['ID=e3248a7c-d6dc-404f-9bb7-8e87d070d1cb,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:59.042691 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-577d8a15-5a1c-4f48-9316-4134a05808c0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.119s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:59.046660 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Request body: {'subnet': {'network_id': 'e3248a7c-d6dc-404f-9bb7-8e87d070d1cb', 'cidr': '172.20.0.0/28', 'name': 'tempest-VolumesActionsTest-108595079-subnet', 'tenant_id': 'c57d56efd5fb40069159f50973655a5d', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:26:59.054038 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:26:59.056023 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Network e3248a7c-d6dc-404f-9bb7-8e87d070d1cb details requested by agent ovs-agent-dv-cinder-782405-9-iscsi with host dv-cinder-782405-9-iscsi {{(pid=91645) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 20 04:26:59.060518 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Usage tracker for resource:subnet and tenant:c57d56efd5fb40069159f50973655a5d is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:26:59.065055 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:26:59.078848 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:26:59.098964 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Made reservation on behalf of c57d56efd5fb40069159f50973655a5d for: {'subnet': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:26:59.102311 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-1810190'] for subnet, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:59.654342 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-1988978', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-1810236'] for subnet, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:59.659748 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-c1cee381-6947-40f4-aed7-216aa79f9601 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:26:59.735091 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Pushing event updated for resources: {'Subnet': ['ID=cb4fcf1d-e5d6-469d-83a3-5d90ce39e474,revision_number=0']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:26:59.750187 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:59.750687 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:59.751075 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:59.751541 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:59.751913 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:59.752283 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:59.752634 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:59.753079 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:59.753464 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:59.753866 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:59.754217 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:59.754553 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:26:59.758885 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.099s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:26:59.760011 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Checking segment: {'id': '0d9a1ad5-c1fc-43c7-8293-6032c9456478', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 809, 'network_id': 'e3248a7c-d6dc-404f-9bb7-8e87d070d1cb'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:26:59.760369 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Network hostable DHCP agents. Network: e3248a7c-d6dc-404f-9bb7-8e87d070d1cb, hostable agents: ['fffae25a-f450-4182-8f9e-ac29d143f6c9'], hosted agents: [] {{(pid=91644) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 20 04:26:59.779801 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Network e3248a7c-d6dc-404f-9bb7-8e87d070d1cb is scheduled to be hosted by DHCP agent fffae25a-f450-4182-8f9e-ac29d143f6c9 with binding_index 1 {{(pid=91644) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 20 04:26:59.796441 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-782405-9-iscsi, e3248a7c-d6dc-404f-9bb7-8e87d070d1cb) {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:26:59.805932 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-26fc03f4-c1cd-4a77-8932-8c0960894a7b None None] Network e3248a7c-d6dc-404f-9bb7-8e87d070d1cb requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:26:59.811443 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:26:59.812039 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Notify callbacks [] for subnet, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:26:59.857404 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] 10.222.0.39 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 845 time: 0.8198793 May 20 04:27:00.006585 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-756c9b58-9179-4473-bfa7-7a5d39985c46 None None] Network e3248a7c-d6dc-404f-9bb7-8e87d070d1cb requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:27:00.126308 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1bc73551-585e-4dac-8e7c-1ea2b7736e2c req-9fce60c1-b9ed-4ea2-832a-b9d4a74282b5 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:00.128082 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1bc73551-585e-4dac-8e7c-1ea2b7736e2c req-9fce60c1-b9ed-4ea2-832a-b9d4a74282b5 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.0843811 May 20 04:27:00.177477 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1bc73551-585e-4dac-8e7c-1ea2b7736e2c req-c74902c7-c8ac-4e8a-848b-b105c5526cbb tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0428770 May 20 04:27:01.398108 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d5dc9c1d-d0f1-44fb-83b8-97abe3a45086 req-883f8363-1f01-4d19-a4db-48ed8ed95f0c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:01.399482 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d5dc9c1d-d0f1-44fb-83b8-97abe3a45086 req-883f8363-1f01-4d19-a4db-48ed8ed95f0c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.0928411 May 20 04:27:01.447207 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d5dc9c1d-d0f1-44fb-83b8-97abe3a45086 req-0c8f3ed7-206b-4df2-8390-42043e3c924c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0388508 May 20 04:27:01.854912 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-cf718572-2646-4058-8a66-8be6d3a6ea96 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:01.855605 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-cf718572-2646-4058-8a66-8be6d3a6ea96 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Synchronizing usage tracker for tenant:c57d56efd5fb40069159f50973655a5d on resource:network {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:27:01.875954 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-cf718572-2646-4058-8a66-8be6d3a6ea96 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Unset dirty status for tenant:c57d56efd5fb40069159f50973655a5d on resource:network {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:27:01.877268 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-cf718572-2646-4058-8a66-8be6d3a6ea96 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/networks?id=e3248a7c-d6dc-404f-9bb7-8e87d070d1cb HTTP/1.1" status: 200 len: 808 time: 0.1665180 May 20 04:27:01.892138 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-f7019416-8c1b-4bd0-b787-609d4f59e44e tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/quotas/c57d56efd5fb40069159f50973655a5d HTTP/1.1" status: 200 len: 375 time: 0.0092688 May 20 04:27:01.947137 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-9cfabf36-ad65-4ae5-8dc4-38e89d72c026 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=c57d56efd5fb40069159f50973655a5d&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0499156 May 20 04:27:02.138719 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-41b99608-8a8a-41a9-9c03-8d4e460799ab service neutron] 10.222.0.39 "GET /v2.0/networks/e3248a7c-d6dc-404f-9bb7-8e87d070d1cb?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1834385 May 20 04:27:02.290277 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-48f4c344-28c2-4c8a-9977-883222446f2c service neutron] 10.222.0.39 "GET /v2.0/networks/e3248a7c-d6dc-404f-9bb7-8e87d070d1cb?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1429260 May 20 04:27:02.456009 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-99345e87-88e3-4a3f-a5a8-eccf610c054e req-a1fefed4-3b5d-4f97-9597-3190781bf29a tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 210 time: 0.0415492 May 20 04:27:02.671876 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6676cc2e-0845-4c83-a72b-f20ebf8eccce req-f06a9c04-d85d-40fd-a62a-83d460e84178 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:02.674376 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6676cc2e-0845-4c83-a72b-f20ebf8eccce req-f06a9c04-d85d-40fd-a62a-83d460e84178 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.1162384 May 20 04:27:02.753360 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6676cc2e-0845-4c83-a72b-f20ebf8eccce req-e4ae75cf-138e-4828-83c2-65abccb4ec76 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0698819 May 20 04:27:03.604462 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a9cf0d94-db16-4d11-82c7-c0333ef1a833 req-fbefe6c0-9ec2-4f09-bcf0-d7a6e5afe8a1 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 210 time: 0.0422060 May 20 04:27:03.963970 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c4a2d0b8-0acf-4ff4-8b46-0d285d7bd699 req-dac0fe8f-2246-46e5-be9d-819858ca480d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:03.965419 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c4a2d0b8-0acf-4ff4-8b46-0d285d7bd699 req-dac0fe8f-2246-46e5-be9d-819858ca480d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.0797629 May 20 04:27:04.012608 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c4a2d0b8-0acf-4ff4-8b46-0d285d7bd699 req-7ee54070-9626-428f-a727-84581c3464db tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0410225 May 20 04:27:04.761531 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9e4edbeb-a12c-439e-91a4-1678bab24a3a req-84de5d70-4076-42f2-b353-38609f29b94c tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 210 time: 0.0530849 May 20 04:27:05.227112 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 40314) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:27:05.271292 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5f5b1d6-ef61-4ea3-8f8a-75c7542b0ad0 req-b95d2fe6-79f9-4ad4-bf4b-87e624b58038 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:05.272742 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b5f5b1d6-ef61-4ea3-8f8a-75c7542b0ad0 req-b95d2fe6-79f9-4ad4-bf4b-87e624b58038 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.0816188 May 20 04:27:05.318379 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b5f5b1d6-ef61-4ea3-8f8a-75c7542b0ad0 req-1b21b482-d35e-4a01-b411-1b408cb170cf tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0399051 May 20 04:27:05.347077 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-f462a960-5514-4350-82cc-a09159d861fd admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:05.363522 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-f462a960-5514-4350-82cc-a09159d861fd admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:27:05.368135 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-f462a960-5514-4350-82cc-a09159d861fd admin admin] Usage tracker for resource:security_group_rule and tenant:c3dd46f7573e46099c4ffdfa63f84156 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:27:05.371747 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-f462a960-5514-4350-82cc-a09159d861fd admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:27:05.378667 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-f462a960-5514-4350-82cc-a09159d861fd admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:27:05.487582 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-f462a960-5514-4350-82cc-a09159d861fd admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:05.492634 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-f462a960-5514-4350-82cc-a09159d861fd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:05.501246 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-abe4ad8b-a054-499c-bb0d-60c87a7120f5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:05.539019 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f462a960-5514-4350-82cc-a09159d861fd admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=6c46fc37-0555-4a7f-8e1b-bb41af493c46,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:05.543115 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-f462a960-5514-4350-82cc-a09159d861fd admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=c3dd46f7573e46099c4ffdfa63f84156&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3151278 May 20 04:27:05.544152 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-f462a960-5514-4350-82cc-a09159d861fd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:05.545862 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 40320) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:27:05.586246 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:05.610571 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9b2e55ca-6643-450d-a78d-bdb390c2d957 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:05.636026 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9b2e55ca-6643-450d-a78d-bdb390c2d957 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:05.697048 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9b2e55ca-6643-450d-a78d-bdb390c2d957 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:05.701773 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-4b224c26-4ff6-462f-92fb-1c7eb49ecb78 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:05.719743 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9b2e55ca-6643-450d-a78d-bdb390c2d957 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=6c46fc37-0555-4a7f-8e1b-bb41af493c46,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:05.722789 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-9b2e55ca-6643-450d-a78d-bdb390c2d957 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.021s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:05.725711 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-9b2e55ca-6643-450d-a78d-bdb390c2d957 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:05.729888 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-9b2e55ca-6643-450d-a78d-bdb390c2d957 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/6c46fc37-0555-4a7f-8e1b-bb41af493c46 HTTP/1.1" status: 204 len: 168 time: 0.1807482 May 20 04:27:05.919844 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-47226dfb-3488-4704-8d39-6b7b04d1eb32 req-06738ef0-63d7-418d-b9bb-0b10c0e8e088 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 210 time: 0.0457771 May 20 04:27:06.064107 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 40330) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:27:06.081576 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-57f2af77-dfdb-42b2-b2bb-080e40601a45 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:06.096463 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-57f2af77-dfdb-42b2-b2bb-080e40601a45 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:27:06.100407 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-57f2af77-dfdb-42b2-b2bb-080e40601a45 admin admin] Usage tracker for resource:security_group_rule and tenant:2131f0c892824579a195f10f256a3f8d is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:27:06.103182 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-57f2af77-dfdb-42b2-b2bb-080e40601a45 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:27:06.108896 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-57f2af77-dfdb-42b2-b2bb-080e40601a45 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:27:06.193027 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-57f2af77-dfdb-42b2-b2bb-080e40601a45 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:06.199313 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-57f2af77-dfdb-42b2-b2bb-080e40601a45 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:06.206471 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9c0683ef-dc4f-42c5-aa84-f40f8badd25d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:06.248147 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-57f2af77-dfdb-42b2-b2bb-080e40601a45 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=2131f0c892824579a195f10f256a3f8d&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1828885 May 20 04:27:06.249062 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-57f2af77-dfdb-42b2-b2bb-080e40601a45 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=8ec1ab8a-6f37-4cc2-8cbe-47e1b4d7b33c,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:06.251884 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 40334) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:27:06.252438 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-57f2af77-dfdb-42b2-b2bb-080e40601a45 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:06.315792 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-598520ef-dd00-45b8-89ed-1511d26b75ac admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:06.342721 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-598520ef-dd00-45b8-89ed-1511d26b75ac admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:06.394653 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-598520ef-dd00-45b8-89ed-1511d26b75ac admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:06.398669 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-a3769a15-b281-4c70-a7e3-bf3c02777a2d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:06.409885 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-598520ef-dd00-45b8-89ed-1511d26b75ac admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=8ec1ab8a-6f37-4cc2-8cbe-47e1b4d7b33c,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:06.412139 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-598520ef-dd00-45b8-89ed-1511d26b75ac admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.014s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:06.417525 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-598520ef-dd00-45b8-89ed-1511d26b75ac admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:06.419061 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-598520ef-dd00-45b8-89ed-1511d26b75ac admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/8ec1ab8a-6f37-4cc2-8cbe-47e1b4d7b33c HTTP/1.1" status: 204 len: 168 time: 0.1652143 May 20 04:27:06.650424 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-be24fbc1-51f3-42ee-9803-73d5d6cad90b req-3b4ff60a-3cc1-4cd6-b2d2-742454c09add tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:06.652241 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-be24fbc1-51f3-42ee-9803-73d5d6cad90b req-3b4ff60a-3cc1-4cd6-b2d2-742454c09add tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.2195978 May 20 04:27:06.699381 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-be24fbc1-51f3-42ee-9803-73d5d6cad90b req-fd4da254-7534-4ee3-804b-6007c0e8e5c9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0406430 May 20 04:27:06.819376 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 40344) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:27:06.837096 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5b6cb960-1705-4ee6-bb83-e44e6d72e38e admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:06.855500 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-5b6cb960-1705-4ee6-bb83-e44e6d72e38e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:27:06.859497 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-5b6cb960-1705-4ee6-bb83-e44e6d72e38e admin admin] Usage tracker for resource:security_group_rule and tenant:ff65dc7281a74a71b6682ac94c09dfe2 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:27:06.862337 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-5b6cb960-1705-4ee6-bb83-e44e6d72e38e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:27:06.868370 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-5b6cb960-1705-4ee6-bb83-e44e6d72e38e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:27:06.973568 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5b6cb960-1705-4ee6-bb83-e44e6d72e38e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:06.981699 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-5b6cb960-1705-4ee6-bb83-e44e6d72e38e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:06.995975 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-3f44364d-ba24-4615-a326-ff5666cb612a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:07.046369 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-5b6cb960-1705-4ee6-bb83-e44e6d72e38e admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=ff65dc7281a74a71b6682ac94c09dfe2&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2235906 May 20 04:27:07.046804 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 40350) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:27:07.075106 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b6cb960-1705-4ee6-bb83-e44e6d72e38e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=2fe0851f-9bfc-4f45-aa4b-3420049aea9d,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:07.088192 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-5b6cb960-1705-4ee6-bb83-e44e6d72e38e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.092s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:07.101061 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-da85c2bc-63f4-4e67-a742-675c7edcbeec req-55e8d34d-fcd0-4f28-89d7-4bd8709e724b tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 210 time: 0.0603652 May 20 04:27:07.117130 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-62c2fe2a-24ac-43c3-8a93-9db183581047 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:07.143357 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-62c2fe2a-24ac-43c3-8a93-9db183581047 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:07.201027 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-62c2fe2a-24ac-43c3-8a93-9db183581047 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:07.205774 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-77babf16-6ab8-45b4-b32a-edf6cc347ed8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:07.215719 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-62c2fe2a-24ac-43c3-8a93-9db183581047 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=2fe0851f-9bfc-4f45-aa4b-3420049aea9d,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:07.220414 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-62c2fe2a-24ac-43c3-8a93-9db183581047 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:07.222436 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-62c2fe2a-24ac-43c3-8a93-9db183581047 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:07.223553 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-62c2fe2a-24ac-43c3-8a93-9db183581047 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/2fe0851f-9bfc-4f45-aa4b-3420049aea9d HTTP/1.1" status: 204 len: 168 time: 0.1756828 May 20 04:27:07.944451 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bf49efe9-684a-4745-bfb7-f1422c3372df req-99648b78-4dd9-43ec-9eef-c6ca0af7c582 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:07.946174 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bf49efe9-684a-4745-bfb7-f1422c3372df req-99648b78-4dd9-43ec-9eef-c6ca0af7c582 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.1084044 May 20 04:27:07.991779 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bf49efe9-684a-4745-bfb7-f1422c3372df req-59e9f318-f4fd-4efb-a24b-fd98b5d2f097 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0395494 May 20 04:27:08.296766 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-be1fc992-4a42-4a2d-b1a1-1bb514c9c09d req-f7a7c16a-9131-4081-8a64-ea15c7e1814d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 210 time: 0.0437911 May 20 04:27:09.204471 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c3be5ce3-5f8e-427d-816f-abd527f51a62 req-2946d9be-24af-43a3-bf0e-46d7909e0124 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:09.211458 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c3be5ce3-5f8e-427d-816f-abd527f51a62 req-2946d9be-24af-43a3-bf0e-46d7909e0124 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.0980425 May 20 04:27:09.254971 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c3be5ce3-5f8e-427d-816f-abd527f51a62 req-c1bda54a-61a9-4e8a-b709-c42d439fdfe9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0425768 May 20 04:27:09.451718 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f00f5f50-d14c-439f-b36a-c7a1d2947b60 req-0625d741-c8c0-4242-b1ec-6245b25f43ea tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 210 time: 0.0473328 May 20 04:27:09.959811 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-17745166-864c-43f5-a827-06188a393239 req-7b73bbc8-3733-4441-a874-c7f3935e88e0 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/networks?tenant_id=0484029d05714b4d860a9e429b5e7d70&shared=False HTTP/1.1" status: 200 len: 213 time: 0.0275419 May 20 04:27:10.128764 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-17745166-864c-43f5-a827-06188a393239 req-d85f2a98-5787-47a0-b1ee-c60d682e92b6 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:10.130219 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-17745166-864c-43f5-a827-06188a393239 req-d85f2a98-5787-47a0-b1ee-c60d682e92b6 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 769 time: 0.1650412 May 20 04:27:10.145779 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-17745166-864c-43f5-a827-06188a393239 req-8ac7a012-0b3f-431c-9911-c6685dc5e180 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/quotas/0484029d05714b4d860a9e429b5e7d70 HTTP/1.1" status: 200 len: 375 time: 0.0098617 May 20 04:27:10.199759 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-17745166-864c-43f5-a827-06188a393239 req-ed7a20e6-56ad-4799-9740-d030048d4d24 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=0484029d05714b4d860a9e429b5e7d70&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0486701 May 20 04:27:10.456545 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f4980954-7c2e-4e09-ab01-9aac9e320748 req-55c0c3ec-891c-4a49-a594-32aedbf9d69d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:10.457985 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f4980954-7c2e-4e09-ab01-9aac9e320748 req-55c0c3ec-891c-4a49-a594-32aedbf9d69d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.0802910 May 20 04:27:10.508766 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f4980954-7c2e-4e09-ab01-9aac9e320748 req-ec0e62d9-c251-4b69-957d-db6e19a9680e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0449741 May 20 04:27:10.609653 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c2ca562d-40fb-41ff-8e50-2e7fc5b4039d req-c647214d-1329-4de2-a067-730c720aa43b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 210 time: 0.0452962 May 20 04:27:10.753122 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8d51866e-0623-47ad-befb-fed653659cdb req-6a92d9b9-6a9a-446c-97c4-ae97dfbe88b7 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 210 time: 0.0477598 May 20 04:27:11.715165 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-37e9fb7f-1cea-47e9-a24e-fdf41692e4a5 req-e53128b5-01a5-4de8-a420-47c2a88ef015 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:11.716619 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-37e9fb7f-1cea-47e9-a24e-fdf41692e4a5 req-e53128b5-01a5-4de8-a420-47c2a88ef015 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.0800157 May 20 04:27:11.760161 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-37e9fb7f-1cea-47e9-a24e-fdf41692e4a5 req-3b4b3435-6615-47f7-a0ce-b22de10d141e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0374603 May 20 04:27:11.918214 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b041c0fe-a833-4741-bf5c-5c13644036f5 req-a92c1954-6a09-4376-82b4-ba4e95d27dfe tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 210 time: 0.0491230 May 20 04:27:12.067572 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d240d307-428e-4ced-9b79-82b64e3a0fcb req-74ef8d36-c898-48ef-aa30-36d094256a3c tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 210 time: 0.0479050 May 20 04:27:12.622415 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-f35f8b19-dd6a-48e1-90a9-f83332ca840a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:12.962421 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ec2afde7-9e9c-4bca-839d-0932f9187b22 req-b15f749c-0496-4bcd-982e-6cb19c5bd90a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:12.963818 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ec2afde7-9e9c-4bca-839d-0932f9187b22 req-b15f749c-0496-4bcd-982e-6cb19c5bd90a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.0839200 May 20 04:27:13.007499 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ec2afde7-9e9c-4bca-839d-0932f9187b22 req-26d7bee9-9e5c-43b4-a993-01180dd4f985 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0376234 May 20 04:27:13.151529 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8522418f-bcf0-487b-b2d7-0d97df298794 req-2ee17bca-109e-4779-b63a-8252b553a864 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 210 time: 0.0429587 May 20 04:27:13.299949 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5684f873-64bb-46c0-9f89-9bb4347c4dc2 req-280dfe67-0b4c-4bc3-a39b-a7998ed7631a tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 210 time: 0.0481007 May 20 04:27:14.288437 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-92c8549e-90d3-4122-9373-a2d622d259e8 req-0ca7b2ee-257e-49a0-ba15-7bfcbbe84015 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:14.291149 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-92c8549e-90d3-4122-9373-a2d622d259e8 req-0ca7b2ee-257e-49a0-ba15-7bfcbbe84015 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.1212215 May 20 04:27:14.379399 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8e19174b-ed8b-4207-a1f0-c36af7a75863 req-a010ae98-0ddf-4c35-ae51-b600144810f2 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 210 time: 0.0599084 May 20 04:27:14.387387 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-92c8549e-90d3-4122-9373-a2d622d259e8 req-4e66b3b8-8729-41fb-864b-65c4afbfe42e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0912993 May 20 04:27:14.519538 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f801cf64-b115-476c-abef-5812804a78e4 req-238e5fd7-764e-4f0c-8585-f27a1f5e8e48 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 210 time: 0.0440164 May 20 04:27:14.799999 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-17745166-864c-43f5-a827-06188a393239 req-5f9eddb0-b1b9-4c26-82bb-7ca784152dad tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.74 "GET /v2.0/networks?tenant_id=0484029d05714b4d860a9e429b5e7d70&shared=False HTTP/1.1" status: 200 len: 213 time: 0.0300169 May 20 04:27:14.981692 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:15.049466 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-17745166-864c-43f5-a827-06188a393239 req-e87a17d5-3804-4ecb-85e4-5fbe76531ec6 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:15.050880 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-17745166-864c-43f5-a827-06188a393239 req-e87a17d5-3804-4ecb-85e4-5fbe76531ec6 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.74 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 769 time: 0.2452085 May 20 04:27:15.066089 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Request body: {'port': {'device_id': '259bb3a0-b99c-43ed-9876-ef59563646c3', 'network_id': '504e252b-5361-4c2a-87b4-bd9946954761', 'admin_state_up': True, 'tenant_id': '0484029d05714b4d860a9e429b5e7d70'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:27:15.090415 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c7f489f3-f0cf-448a-803e-49552f96417f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:15.205196 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:27:15.208985 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Usage tracker for resource:port and tenant:0484029d05714b4d860a9e429b5e7d70 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:27:15.212007 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Quota usage for port was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:27:15.220895 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:27:15.232899 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Made reservation on behalf of 0484029d05714b4d860a9e429b5e7d70 for: {'port': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:27:15.237733 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:15.251343 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:15.266470 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:27:15.270553 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Usage tracker for resource:security_group_rule and tenant:0484029d05714b4d860a9e429b5e7d70 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:27:15.274106 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:27:15.280407 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:27:15.368741 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-e979938e-c8a9-4a5b-be29-89f3eea0f4ed tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:15.370281 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-e979938e-c8a9-4a5b-be29-89f3eea0f4ed tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.74 "GET /v2.0/networks?id=e3248a7c-d6dc-404f-9bb7-8e87d070d1cb HTTP/1.1" status: 200 len: 808 time: 0.1512785 May 20 04:27:15.376807 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:15.381203 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:15.384913 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-cb99be19-b420-4c3a-94ca-8aa7437f6764 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Request body: {'port': {'device_id': '5a913a0a-8f85-45b8-a226-396c3539d204', 'network_id': 'e3248a7c-d6dc-404f-9bb7-8e87d070d1cb', 'admin_state_up': True, 'tenant_id': 'c57d56efd5fb40069159f50973655a5d'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:27:15.427135 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-b7c033a2-8bfe-4a59-b278-426ec5f4b7c0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:15.482000 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Pushing event updated for resources: {'SecurityGroup': ['ID=1b091fdd-0a0f-46fb-8de3-aea8fa6fdde8,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:15.485268 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.058s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:15.544654 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-cb99be19-b420-4c3a-94ca-8aa7437f6764 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:27:15.548664 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-cb99be19-b420-4c3a-94ca-8aa7437f6764 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Usage tracker for resource:port and tenant:c57d56efd5fb40069159f50973655a5d is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:27:15.551658 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-cb99be19-b420-4c3a-94ca-8aa7437f6764 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Quota usage for port was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:27:15.558969 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-cb99be19-b420-4c3a-94ca-8aa7437f6764 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:27:15.571529 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-cb99be19-b420-4c3a-94ca-8aa7437f6764 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Made reservation on behalf of c57d56efd5fb40069159f50973655a5d for: {'port': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:27:15.572480 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Allocated IP 192.168.233.105 (504e252b-5361-4c2a-87b4-bd9946954761/3d544974-6fec-477e-a946-f434a49fa867/04700561-9728-48ff-aa15-d99ae8180704) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:27:15.576109 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-cb99be19-b420-4c3a-94ca-8aa7437f6764 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:15.663337 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9f084ac6-af85-4e35-9c3d-1f21a6b351d0 req-2de50d41-c8fa-43f9-9d2f-fbc29726cddd tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 210 time: 0.0879793 May 20 04:27:15.763983 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-cb99be19-b420-4c3a-94ca-8aa7437f6764 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Allocated IP 172.20.0.8 (e3248a7c-d6dc-404f-9bb7-8e87d070d1cb/cb4fcf1d-e5d6-469d-83a3-5d90ce39e474/dcf04ca1-f449-4868-acab-73152c97c4c8) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:27:15.793549 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d95bd08-7054-4e26-8e5c-8d5ed54af70d req-c891ccc6-307b-4768-8735-68117b743236 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:15.795021 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1d95bd08-7054-4e26-8e5c-8d5ed54af70d req-c891ccc6-307b-4768-8735-68117b743236 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.1800008 May 20 04:27:15.926802 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d68282ef-1a97-4434-b971-b5bbd4fdbe0f req-67456c66-3ae4-425b-9773-b32150ec4ae7 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 210 time: 0.1193771 May 20 04:27:15.978504 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1d95bd08-7054-4e26-8e5c-8d5ed54af70d req-08809064-29fc-4df9-bd6e-bb056499bd33 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1731982 May 20 04:27:16.017486 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-cb99be19-b420-4c3a-94ca-8aa7437f6764 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:16.034020 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:16.050150 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-cb99be19-b420-4c3a-94ca-8aa7437f6764 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:16.052020 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.provisioning_blocks [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Transition to ACTIVE for port object 04700561-9728-48ff-aa15-d99ae8180704 will not be triggered until provisioned by entity DHCP. {{(pid=91643) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:27:16.057781 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:16.081275 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:16.086948 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:16.128671 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-cb99be19-b420-4c3a-94ca-8aa7437f6764 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Network e3248a7c-d6dc-404f-9bb7-8e87d070d1cb is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:27:16.131874 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-cb99be19-b420-4c3a-94ca-8aa7437f6764 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:27:16.134417 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-cb99be19-b420-4c3a-94ca-8aa7437f6764 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:27:16.158182 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Network 504e252b-5361-4c2a-87b4-bd9946954761 is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:27:16.161223 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:27:16.165602 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:27:16.175961 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cb99be19-b420-4c3a-94ca-8aa7437f6764 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Pushing event updated for resources: {'Port': ['ID=dcf04ca1-f449-4868-acab-73152c97c4c8,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:16.183705 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-cb99be19-b420-4c3a-94ca-8aa7437f6764 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.126s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:16.191307 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:16.193795 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Pushing event updated for resources: {'Port': ['ID=04700561-9728-48ff-aa15-d99ae8180704,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:16.200926 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:16.207190 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-cb99be19-b420-4c3a-94ca-8aa7437f6764 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:16.207850 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-cb99be19-b420-4c3a-94ca-8aa7437f6764 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:16.230278 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-cb99be19-b420-4c3a-94ca-8aa7437f6764 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 0.8492396 May 20 04:27:16.232851 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:16.233541 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:16.242134 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Request body: {'port': {'device_id': '5a913a0a-8f85-45b8-a226-396c3539d204', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:27:16.271440 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-17745166-864c-43f5-a827-06188a393239 req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 938 time: 1.2088625 May 20 04:27:16.288722 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Request body: {'port': {'device_id': '259bb3a0-b99c-43ed-9876-ef59563646c3', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:27:16.361310 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:16.404542 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:16.460935 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-02e2243c-7b17-4797-8b44-bf5c2e864bb9 None None] Provisioning for port 04700561-9728-48ff-aa15-d99ae8180704 completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:27:16.466600 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-02e2243c-7b17-4797-8b44-bf5c2e864bb9 None None] Provisioning complete for port 04700561-9728-48ff-aa15-d99ae8180704 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:27:16.467321 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-02e2243c-7b17-4797-8b44-bf5c2e864bb9 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:16.522797 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-02e2243c-7b17-4797-8b44-bf5c2e864bb9 None None] Port 04700561-9728-48ff-aa15-d99ae8180704 cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:27:16.762136 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 04700561-9728-48ff-aa15-d99ae8180704 {{(pid=91643) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:27:16.771769 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:16.786759 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id dcf04ca1-f449-4868-acab-73152c97c4c8 {{(pid=91644) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:27:16.793974 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:16.807942 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Transition to ACTIVE for port object dcf04ca1-f449-4868-acab-73152c97c4c8 will not be triggered until provisioned by entity L2. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:27:16.812464 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:16.817100 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-ed6c0aaf-3f7f-4e76-88a6-42d4d6a13b89 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:16.829662 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.provisioning_blocks [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Transition to ACTIVE for port object 04700561-9728-48ff-aa15-d99ae8180704 will not be triggered until provisioned by entity L2. {{(pid=91643) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:27:16.840467 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:16.846687 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:16.950158 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Pushing event updated for resources: {'Port': ['ID=dcf04ca1-f449-4868-acab-73152c97c4c8,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:17.002332 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.185s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:17.101135 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Attempting to bind port dcf04ca1-f449-4868-acab-73152c97c4c8 on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:27:17.101791 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Attempting to bind port dcf04ca1-f449-4868-acab-73152c97c4c8 by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': '0d9a1ad5-c1fc-43c7-8293-6032c9456478', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 809, 'network_id': 'e3248a7c-d6dc-404f-9bb7-8e87d070d1cb'}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:27:17.102255 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Attempting to bind port dcf04ca1-f449-4868-acab-73152c97c4c8 on network e3248a7c-d6dc-404f-9bb7-8e87d070d1cb {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:27:17.107197 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Port dcf04ca1-f449-4868-acab-73152c97c4c8 on network e3248a7c-d6dc-404f-9bb7-8e87d070d1cb not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:27:17.107809 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Attempting to bind port dcf04ca1-f449-4868-acab-73152c97c4c8 on network e3248a7c-d6dc-404f-9bb7-8e87d070d1cb {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:27:17.113054 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 27, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:27:17.113441 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Checking segment: {'id': '0d9a1ad5-c1fc-43c7-8293-6032c9456478', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 809, 'network_id': 'e3248a7c-d6dc-404f-9bb7-8e87d070d1cb'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:27:17.113875 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Bound using segment: {'id': '0d9a1ad5-c1fc-43c7-8293-6032c9456478', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 809, 'network_id': 'e3248a7c-d6dc-404f-9bb7-8e87d070d1cb'} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:27:17.114369 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Bound port: dcf04ca1-f449-4868-acab-73152c97c4c8, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '0d9a1ad5-c1fc-43c7-8293-6032c9456478', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 809, 'network_id': 'e3248a7c-d6dc-404f-9bb7-8e87d070d1cb'}}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:27:17.114769 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:17.159675 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Pushing event updated for resources: {'Port': ['ID=04700561-9728-48ff-aa15-d99ae8180704,revision_number=2']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:17.162554 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.316s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:17.207531 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] For port dcf04ca1-f449-4868-acab-73152c97c4c8, host hv-cinder-78240, cleared binding levels {{(pid=91644) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:27:17.210864 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] For port dcf04ca1-f449-4868-acab-73152c97c4c8, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=dcf04ca1-f449-4868-acab-73152c97c4c8,segment=NetworkSegment(0d9a1ad5-c1fc-43c7-8293-6032c9456478),segment_id=0d9a1ad5-c1fc-43c7-8293-6032c9456478)] {{(pid=91644) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:27:17.217757 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:17.223018 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-67649c1d-2603-4b32-8769-3eaad9392d10 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:17.249199 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cb602b98-b2c7-43b9-92b8-d3b70cfccd88 req-714b82d9-80b1-4474-8a95-37d53cbcc286 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:17.249894 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-cb602b98-b2c7-43b9-92b8-d3b70cfccd88 req-714b82d9-80b1-4474-8a95-37d53cbcc286 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Synchronizing usage tracker for tenant:0484029d05714b4d860a9e429b5e7d70 on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:27:17.277009 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:17.277618 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:17.278966 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] 10.222.0.74 "PUT /v2.0/ports/dcf04ca1-f449-4868-acab-73152c97c4c8 HTTP/1.1" status: 200 len: 1093 time: 1.0413713 May 20 04:27:17.279489 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '5a913a0a-8f85-45b8-a226-396c3539d204', 'tag': 'dcf04ca1-f449-4868-acab-73152c97c4c8'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:27:17.300557 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Pushing event updated for resources: {'Port': ['ID=dcf04ca1-f449-4868-acab-73152c97c4c8,revision_number=3']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:17.303418 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.080s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:17.320263 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Attempting to bind port 04700561-9728-48ff-aa15-d99ae8180704 on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:27:17.320602 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Attempting to bind port 04700561-9728-48ff-aa15-d99ae8180704 by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': '0fbdfc2a-8204-4d65-bf3c-9044da0482d3', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 816, 'network_id': '504e252b-5361-4c2a-87b4-bd9946954761'}] {{(pid=91643) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:27:17.320936 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Attempting to bind port 04700561-9728-48ff-aa15-d99ae8180704 on network 504e252b-5361-4c2a-87b4-bd9946954761 {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:27:17.323871 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-cb602b98-b2c7-43b9-92b8-d3b70cfccd88 req-714b82d9-80b1-4474-8a95-37d53cbcc286 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Unset dirty status for tenant:0484029d05714b4d860a9e429b5e7d70 on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:27:17.325009 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cb602b98-b2c7-43b9-92b8-d3b70cfccd88 req-714b82d9-80b1-4474-8a95-37d53cbcc286 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 936 time: 0.5490055 May 20 04:27:17.330514 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Port 04700561-9728-48ff-aa15-d99ae8180704 on network 504e252b-5361-4c2a-87b4-bd9946954761 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:27:17.330838 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Attempting to bind port 04700561-9728-48ff-aa15-d99ae8180704 on network 504e252b-5361-4c2a-87b4-bd9946954761 {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:27:17.342752 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-152ca65c-dea3-4fc8-b6cb-a99cb51e1318 req-6011f563-cad9-4998-bdec-cc10e4289f3d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:17.344223 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-152ca65c-dea3-4fc8-b6cb-a99cb51e1318 req-6011f563-cad9-4998-bdec-cc10e4289f3d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 943 time: 0.2498178 May 20 04:27:17.358523 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 27, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:27:17.358864 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Checking segment: {'id': '0fbdfc2a-8204-4d65-bf3c-9044da0482d3', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 816, 'network_id': '504e252b-5361-4c2a-87b4-bd9946954761'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91643) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:27:17.359261 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Bound using segment: {'id': '0fbdfc2a-8204-4d65-bf3c-9044da0482d3', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 816, 'network_id': '504e252b-5361-4c2a-87b4-bd9946954761'} {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:27:17.359744 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Bound port: 04700561-9728-48ff-aa15-d99ae8180704, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '0fbdfc2a-8204-4d65-bf3c-9044da0482d3', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 816, 'network_id': '504e252b-5361-4c2a-87b4-bd9946954761'}}] {{(pid=91643) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:27:17.360089 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:17.394420 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-590f4d3a-da03-4c37-9301-e3ae91ea6376 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:17.395987 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-590f4d3a-da03-4c37-9301-e3ae91ea6376 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=c57d56efd5fb40069159f50973655a5d&device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 1096 time: 0.0757859 May 20 04:27:17.465761 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-07e7226a-c5a4-48b7-aa44-97c60a7d8a18 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=dcf04ca1-f449-4868-acab-73152c97c4c8 HTTP/1.1" status: 200 len: 217 time: 0.0621145 May 20 04:27:17.470191 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-cb602b98-b2c7-43b9-92b8-d3b70cfccd88 req-8aa83237-e8ad-4254-a2d0-6eac0dbcb82c tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Synchronizing usage tracker for tenant:0484029d05714b4d860a9e429b5e7d70 on resource:security_group {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:27:17.482342 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-152ca65c-dea3-4fc8-b6cb-a99cb51e1318 req-f3c61916-5889-485d-861e-040f142dcf6a tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Synchronizing usage tracker for tenant:c57d56efd5fb40069159f50973655a5d on resource:security_group {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:27:17.527241 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-cb602b98-b2c7-43b9-92b8-d3b70cfccd88 req-8aa83237-e8ad-4254-a2d0-6eac0dbcb82c tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Unset dirty status for tenant:0484029d05714b4d860a9e429b5e7d70 on resource:security_group {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:27:17.528434 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cb602b98-b2c7-43b9-92b8-d3b70cfccd88 req-8aa83237-e8ad-4254-a2d0-6eac0dbcb82c tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/security-groups?id=1b091fdd-0a0f-46fb-8de3-aea8fa6fdde8&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1939361 May 20 04:27:17.528784 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-49c74320-c479-4579-aa84-2dd1f0124296 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:17.530210 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-49c74320-c479-4579-aa84-2dd1f0124296 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=cb4fcf1d-e5d6-469d-83a3-5d90ce39e474 HTTP/1.1" status: 200 len: 848 time: 0.0577426 May 20 04:27:17.533185 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] For port 04700561-9728-48ff-aa15-d99ae8180704, host hv-cinder-78240, cleared binding levels {{(pid=91643) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:27:17.536723 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] For port 04700561-9728-48ff-aa15-d99ae8180704, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=04700561-9728-48ff-aa15-d99ae8180704,segment=NetworkSegment(0fbdfc2a-8204-4d65-bf3c-9044da0482d3),segment_id=0fbdfc2a-8204-4d65-bf3c-9044da0482d3)] {{(pid=91643) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:27:17.537727 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-152ca65c-dea3-4fc8-b6cb-a99cb51e1318 req-f3c61916-5889-485d-861e-040f142dcf6a tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Unset dirty status for tenant:c57d56efd5fb40069159f50973655a5d on resource:security_group {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:27:17.539110 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-152ca65c-dea3-4fc8-b6cb-a99cb51e1318 req-f3c61916-5889-485d-861e-040f142dcf6a tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1690123 May 20 04:27:17.542416 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:17.548992 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:17.582061 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-404dc70e-da13-49b0-936c-de835cd70184 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=e3248a7c-d6dc-404f-9bb7-8e87d070d1cb&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0442588 May 20 04:27:17.612188 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:17.612801 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:17.614568 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-17745166-864c-43f5-a827-06188a393239 req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] 10.222.0.74 "PUT /v2.0/ports/04700561-9728-48ff-aa15-d99ae8180704 HTTP/1.1" status: 200 len: 1098 time: 1.3303182 May 20 04:27:17.617609 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '259bb3a0-b99c-43ed-9876-ef59563646c3', 'tag': '04700561-9728-48ff-aa15-d99ae8180704'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:27:17.763874 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-437913e6-f4be-468c-8446-aae5c9da6c3d service neutron] 10.222.0.74 "GET /v2.0/networks/e3248a7c-d6dc-404f-9bb7-8e87d070d1cb?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1742444 May 20 04:27:17.767775 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Pushing event updated for resources: {'Port': ['ID=04700561-9728-48ff-aa15-d99ae8180704,revision_number=3']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:17.779911 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.231s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:17.834381 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:17.839955 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7355464e-3fbd-493d-9857-085cc30b5635 req-67d44381-bcc6-4a90-bea7-293dd6ef2f4b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:17.842215 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7355464e-3fbd-493d-9857-085cc30b5635 req-67d44381-bcc6-4a90-bea7-293dd6ef2f4b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.1277752 May 20 04:27:17.844283 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-17745166-864c-43f5-a827-06188a393239 req-91f37dea-6537-4b1b-8e0d-c458e92693f4 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:17.845780 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-17745166-864c-43f5-a827-06188a393239 req-91f37dea-6537-4b1b-8e0d-c458e92693f4 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=0484029d05714b4d860a9e429b5e7d70&device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 1101 time: 0.1913486 May 20 04:27:17.930333 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-795429eb-7faf-40c2-9d42-743d9c66b4aa req-c5c3ddef-e0e1-4654-9eda-cf0c1fb1faee service neutron] 10.222.0.74 "GET /v2.0/networks/e3248a7c-d6dc-404f-9bb7-8e87d070d1cb?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1581347 May 20 04:27:17.932921 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-17745166-864c-43f5-a827-06188a393239 req-3659095e-f650-4e26-8b1f-ac59aac03748 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=192.168.233.105&port_id=04700561-9728-48ff-aa15-d99ae8180704 HTTP/1.1" status: 200 len: 217 time: 0.0648491 May 20 04:27:18.003048 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7355464e-3fbd-493d-9857-085cc30b5635 req-e5142323-adc3-4aa2-bdac-a44e52b02198 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1540859 May 20 04:27:18.018796 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-17745166-864c-43f5-a827-06188a393239 req-c60ee2ff-b946-4d43-875b-78c047a7cb38 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:18.020283 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-17745166-864c-43f5-a827-06188a393239 req-c60ee2ff-b946-4d43-875b-78c047a7cb38 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=3d544974-6fec-477e-a946-f434a49fa867 HTTP/1.1" status: 200 len: 843 time: 0.0770845 May 20 04:27:18.134629 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-17745166-864c-43f5-a827-06188a393239 req-bd14bb67-c045-4bb9-8a11-d9cf2bc4eafc service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:18.136272 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-17745166-864c-43f5-a827-06188a393239 req-bd14bb67-c045-4bb9-8a11-d9cf2bc4eafc service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=504e252b-5361-4c2a-87b4-bd9946954761&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1178 time: 0.1056695 May 20 04:27:18.251127 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '259bb3a0-b99c-43ed-9876-ef59563646c3', 'tag': '04700561-9728-48ff-aa15-d99ae8180704', 'status': 'completed', 'code': 200} May 20 04:27:18.257030 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '5a913a0a-8f85-45b8-a226-396c3539d204', 'tag': 'dcf04ca1-f449-4868-acab-73152c97c4c8', 'status': 'completed', 'code': 200} May 20 04:27:18.283396 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-17745166-864c-43f5-a827-06188a393239 req-8a727e61-2ccd-4a8f-a439-0d3035dec35e service neutron] 10.222.0.74 "GET /v2.0/networks/504e252b-5361-4c2a-87b4-bd9946954761?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1385112 May 20 04:27:18.353136 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c92e72d2-4e80-4858-ae12-f93b3795f658 req-2395bd17-ca95-4083-acae-9f5268c931b2 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:18.354542 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c92e72d2-4e80-4858-ae12-f93b3795f658 req-2395bd17-ca95-4083-acae-9f5268c931b2 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=c57d56efd5fb40069159f50973655a5d&device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 1096 time: 0.0758219 May 20 04:27:18.415592 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-17745166-864c-43f5-a827-06188a393239 req-d92a54fa-194a-44ac-ab70-697b08b9e6ed service neutron] 10.222.0.74 "GET /v2.0/networks/504e252b-5361-4c2a-87b4-bd9946954761?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1258101 May 20 04:27:18.487867 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c92e72d2-4e80-4858-ae12-f93b3795f658 req-07dcf804-2fae-42b2-80bb-d1491cb4f301 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:18.489203 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c92e72d2-4e80-4858-ae12-f93b3795f658 req-07dcf804-2fae-42b2-80bb-d1491cb4f301 service neutron] 10.222.0.74 "GET /v2.0/networks?id=e3248a7c-d6dc-404f-9bb7-8e87d070d1cb HTTP/1.1" status: 200 len: 909 time: 0.1277149 May 20 04:27:18.552451 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4c30e255-0f0e-471a-9578-0a9bd8a4dbfa req-d21f9bfa-2579-49ce-a7b9-0512b841a6b9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:18.554070 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4c30e255-0f0e-471a-9578-0a9bd8a4dbfa req-d21f9bfa-2579-49ce-a7b9-0512b841a6b9 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=0484029d05714b4d860a9e429b5e7d70&device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 1101 time: 0.0756936 May 20 04:27:18.554852 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c92e72d2-4e80-4858-ae12-f93b3795f658 req-460f17ba-90ec-41bb-9afc-6d30af88ce80 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=dcf04ca1-f449-4868-acab-73152c97c4c8 HTTP/1.1" status: 200 len: 217 time: 0.0583904 May 20 04:27:18.727704 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c92e72d2-4e80-4858-ae12-f93b3795f658 req-d0ab4cab-a412-4a33-b556-21cd5c84171d service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:18.730026 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c92e72d2-4e80-4858-ae12-f93b3795f658 req-d0ab4cab-a412-4a33-b556-21cd5c84171d service neutron] 10.222.0.74 "GET /v2.0/subnets?id=cb4fcf1d-e5d6-469d-83a3-5d90ce39e474 HTTP/1.1" status: 200 len: 848 time: 0.1618166 May 20 04:27:18.778653 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c92e72d2-4e80-4858-ae12-f93b3795f658 req-5ed0fe45-41c1-44af-ae7d-c9b52b46c0d7 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=e3248a7c-d6dc-404f-9bb7-8e87d070d1cb&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0404437 May 20 04:27:18.809706 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4c30e255-0f0e-471a-9578-0a9bd8a4dbfa req-7148f69b-1e42-4cec-93ee-b6aec7db0e7e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:18.811120 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4c30e255-0f0e-471a-9578-0a9bd8a4dbfa req-7148f69b-1e42-4cec-93ee-b6aec7db0e7e service neutron] 10.222.0.74 "GET /v2.0/networks?id=504e252b-5361-4c2a-87b4-bd9946954761 HTTP/1.1" status: 200 len: 870 time: 0.2440820 May 20 04:27:18.884734 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4c30e255-0f0e-471a-9578-0a9bd8a4dbfa req-ee3fa570-2085-46e4-954d-1384b89ee692 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=192.168.233.105&port_id=04700561-9728-48ff-aa15-d99ae8180704 HTTP/1.1" status: 200 len: 217 time: 0.0647242 May 20 04:27:18.894772 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fdc69967-f5b1-407f-ad5e-7f56381100c3 req-80b16d0b-1275-44d5-8108-e6f796ef484c tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:18.896208 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fdc69967-f5b1-407f-ad5e-7f56381100c3 req-80b16d0b-1275-44d5-8108-e6f796ef484c tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 948 time: 0.2415762 May 20 04:27:18.910977 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c92e72d2-4e80-4858-ae12-f93b3795f658 req-ebdd7a49-d7f4-4d82-a9ff-41de71e51c1f service neutron] 10.222.0.74 "GET /v2.0/networks/e3248a7c-d6dc-404f-9bb7-8e87d070d1cb?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1250765 May 20 04:27:19.050931 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c92e72d2-4e80-4858-ae12-f93b3795f658 req-7788b772-389c-4c00-8f19-14c0dfbea180 service neutron] 10.222.0.74 "GET /v2.0/networks/e3248a7c-d6dc-404f-9bb7-8e87d070d1cb?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1330831 May 20 04:27:19.058177 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3588f71b-d52f-48a3-bbda-304e8c1f6a6f req-3753baab-b501-400a-93a9-2c18107ab80f tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:19.059613 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3588f71b-d52f-48a3-bbda-304e8c1f6a6f req-3753baab-b501-400a-93a9-2c18107ab80f tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 943 time: 0.3598824 May 20 04:27:19.124642 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4c30e255-0f0e-471a-9578-0a9bd8a4dbfa req-6af7fff9-13a1-4876-ba07-0c486935272c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:19.126073 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4c30e255-0f0e-471a-9578-0a9bd8a4dbfa req-6af7fff9-13a1-4876-ba07-0c486935272c service neutron] 10.222.0.74 "GET /v2.0/subnets?id=3d544974-6fec-477e-a946-f434a49fa867 HTTP/1.1" status: 200 len: 843 time: 0.2290502 May 20 04:27:19.128795 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fdc69967-f5b1-407f-ad5e-7f56381100c3 req-25d11d44-d3a7-4b21-84ac-d7705d2bffd9 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/security-groups?id=1b091fdd-0a0f-46fb-8de3-aea8fa6fdde8&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1993546 May 20 04:27:19.180135 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3588f71b-d52f-48a3-bbda-304e8c1f6a6f req-17edc068-ed51-4347-b874-4106119ee573 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1113446 May 20 04:27:19.212537 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4c30e255-0f0e-471a-9578-0a9bd8a4dbfa req-ab502062-e0c7-47b2-a1fd-c1171194f38c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:19.214067 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4c30e255-0f0e-471a-9578-0a9bd8a4dbfa req-ab502062-e0c7-47b2-a1fd-c1171194f38c service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=504e252b-5361-4c2a-87b4-bd9946954761&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1178 time: 0.0797505 May 20 04:27:19.402205 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4c30e255-0f0e-471a-9578-0a9bd8a4dbfa req-bc02adb7-6e00-44e7-ad6e-2c8ed7c7934c service neutron] 10.222.0.74 "GET /v2.0/networks/504e252b-5361-4c2a-87b4-bd9946954761?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1798117 May 20 04:27:19.417748 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5923dbeb-04cb-4256-9283-ff7a61e16977 req-67f9da16-5633-4f51-aad6-fee3496b2bfb tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:19.419110 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5923dbeb-04cb-4256-9283-ff7a61e16977 req-67f9da16-5633-4f51-aad6-fee3496b2bfb tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.1758957 May 20 04:27:19.545255 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5923dbeb-04cb-4256-9283-ff7a61e16977 req-2d204b90-828a-486a-a1d9-93f463fc0d35 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1093512 May 20 04:27:19.576970 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4c30e255-0f0e-471a-9578-0a9bd8a4dbfa req-d88e8162-f72a-4c13-bbfb-bc587c97e956 service neutron] 10.222.0.74 "GET /v2.0/networks/504e252b-5361-4c2a-87b4-bd9946954761?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1670721 May 20 04:27:20.376296 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9d5f3ac7-622d-42c6-8e10-8002764ea74b req-872e283c-f1fc-4c14-8cdd-16841cb69987 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:20.377767 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9d5f3ac7-622d-42c6-8e10-8002764ea74b req-872e283c-f1fc-4c14-8cdd-16841cb69987 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 948 time: 0.1409481 May 20 04:27:20.404216 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7aa478c6-5f31-4975-8557-ba6f2bd4cb21 req-6fe1c36c-6ce5-4cb5-a385-1ba0916f7f6c tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:20.405576 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7aa478c6-5f31-4975-8557-ba6f2bd4cb21 req-6fe1c36c-6ce5-4cb5-a385-1ba0916f7f6c tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 943 time: 0.1123948 May 20 04:27:20.454729 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9d5f3ac7-622d-42c6-8e10-8002764ea74b req-63d8e130-4e1c-400e-8bb5-c5820e71e0b6 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/security-groups?id=1b091fdd-0a0f-46fb-8de3-aea8fa6fdde8&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0702024 May 20 04:27:20.466431 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7aa478c6-5f31-4975-8557-ba6f2bd4cb21 req-810fe4c2-b708-4bc7-a2ba-d128261f125b tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0535800 May 20 04:27:20.754281 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-62a690b5-a250-46b0-81af-e46839fcfdc5 req-17fac316-8999-4a0a-bf2e-d56fcadefa32 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:20.755759 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-62a690b5-a250-46b0-81af-e46839fcfdc5 req-17fac316-8999-4a0a-bf2e-d56fcadefa32 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.0852463 May 20 04:27:20.800048 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-62a690b5-a250-46b0-81af-e46839fcfdc5 req-240477d3-eb50-4a12-bb6f-01f23f7de387 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0386033 May 20 04:27:21.702558 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e1a350b4-348c-4d34-894e-b724ac55b22d req-7ce83e55-d390-4749-af28-728fe7054627 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:21.703933 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e1a350b4-348c-4d34-894e-b724ac55b22d req-7ce83e55-d390-4749-af28-728fe7054627 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 948 time: 0.1403785 May 20 04:27:21.714966 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-637de092-e021-4bd5-b81c-245725fd67fb req-8259938f-143d-49ae-bd0c-a98d251061a8 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:21.716348 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-637de092-e021-4bd5-b81c-245725fd67fb req-8259938f-143d-49ae-bd0c-a98d251061a8 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 943 time: 0.1196468 May 20 04:27:21.782064 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e1a350b4-348c-4d34-894e-b724ac55b22d req-4e584c69-f051-476a-b2e7-1ee98cc682c7 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/security-groups?id=1b091fdd-0a0f-46fb-8de3-aea8fa6fdde8&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0644810 May 20 04:27:21.789098 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-637de092-e021-4bd5-b81c-245725fd67fb req-59cc2d29-3352-4172-8089-5def273ee95d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0588923 May 20 04:27:22.006918 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e089e8ab-2d89-4936-bc13-694bf90cf586 req-9b684a85-3a3e-4bf3-88ba-0b17c9d0cbea tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:22.008265 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e089e8ab-2d89-4936-bc13-694bf90cf586 req-9b684a85-3a3e-4bf3-88ba-0b17c9d0cbea tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.0887158 May 20 04:27:22.053434 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e089e8ab-2d89-4936-bc13-694bf90cf586 req-3fae466a-d263-4b88-afc5-c1b61b8f194a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0393109 May 20 04:27:23.034888 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b131bb2f-66f4-4251-823c-b5956547e377 req-9ac3bd1d-2a0e-49e2-a65c-d70f38f7eb00 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:23.036407 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b131bb2f-66f4-4251-823c-b5956547e377 req-9ac3bd1d-2a0e-49e2-a65c-d70f38f7eb00 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 948 time: 0.1374211 May 20 04:27:23.048297 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7f5246b5-7ed5-4f08-8f95-1261d87c27e4 req-fd57f33e-2fac-47fe-9d31-d66ded4f77a3 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:23.049644 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7f5246b5-7ed5-4f08-8f95-1261d87c27e4 req-fd57f33e-2fac-47fe-9d31-d66ded4f77a3 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 943 time: 0.1201463 May 20 04:27:23.116981 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b131bb2f-66f4-4251-823c-b5956547e377 req-cfce10c9-9e83-4217-bfed-95bfa3c78325 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/security-groups?id=1b091fdd-0a0f-46fb-8de3-aea8fa6fdde8&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0664058 May 20 04:27:23.121231 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7f5246b5-7ed5-4f08-8f95-1261d87c27e4 req-c6d0787c-9221-4c4f-9759-95431520dc70 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0583222 May 20 04:27:23.272200 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 04700561-9728-48ff-aa15-d99ae8180704 details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:27:23.302478 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-957d49c4-904c-4bdd-ac5f-87a05cc5f056 req-015186b7-b609-4e4e-a0e9-47748ccee9d7 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:23.303951 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-957d49c4-904c-4bdd-ac5f-87a05cc5f056 req-015186b7-b609-4e4e-a0e9-47748ccee9d7 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.0851545 May 20 04:27:23.348472 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-957d49c4-904c-4bdd-ac5f-87a05cc5f056 req-d274a54f-f97c-4e53-a123-61d7e1801bfc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0386035 May 20 04:27:23.409963 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 04700561-9728-48ff-aa15-d99ae8180704, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=04700561-9728-48ff-aa15-d99ae8180704,segment=NetworkSegment(0fbdfc2a-8204-4d65-bf3c-9044da0482d3),segment_id=0fbdfc2a-8204-4d65-bf3c-9044da0482d3)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:27:23.423353 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': '04700561-9728-48ff-aa15-d99ae8180704', 'network_id': '504e252b-5361-4c2a-87b4-bd9946954761', 'port_id': '04700561-9728-48ff-aa15-d99ae8180704', 'mac_address': 'fa:16:3e:9d:10:27', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 816, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '3d544974-6fec-477e-a946-f434a49fa867', 'ip_address': '192.168.233.105'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:27:23.478377 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:23.495514 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 04700561-9728-48ff-aa15-d99ae8180704 {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:27:23.510960 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 04700561-9728-48ff-aa15-d99ae8180704, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=04700561-9728-48ff-aa15-d99ae8180704,segment=NetworkSegment(0fbdfc2a-8204-4d65-bf3c-9044da0482d3),segment_id=0fbdfc2a-8204-4d65-bf3c-9044da0482d3)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:27:23.516992 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:23.517693 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device dcf04ca1-f449-4868-acab-73152c97c4c8 details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:27:23.519618 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:23.615140 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=04700561-9728-48ff-aa15-d99ae8180704,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:23.666943 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:27:23.667318 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:27:23.667650 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:27:23.667975 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:27:23.668294 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:27:23.668617 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:27:23.668908 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:27:23.669224 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:27:23.669542 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:27:23.669854 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:27:23.670163 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:27:23.670472 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:27:23.675678 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.156s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:23.718838 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port dcf04ca1-f449-4868-acab-73152c97c4c8, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=dcf04ca1-f449-4868-acab-73152c97c4c8,segment=NetworkSegment(0d9a1ad5-c1fc-43c7-8293-6032c9456478),segment_id=0d9a1ad5-c1fc-43c7-8293-6032c9456478)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:27:23.756136 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': 'dcf04ca1-f449-4868-acab-73152c97c4c8', 'network_id': 'e3248a7c-d6dc-404f-9bb7-8e87d070d1cb', 'port_id': 'dcf04ca1-f449-4868-acab-73152c97c4c8', 'mac_address': 'fa:16:3e:e2:a7:1d', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 809, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'cb4fcf1d-e5d6-469d-83a3-5d90ce39e474', 'ip_address': '172.20.0.8'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:27:23.809384 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:23.825947 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id dcf04ca1-f449-4868-acab-73152c97c4c8 {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:27:23.842738 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port dcf04ca1-f449-4868-acab-73152c97c4c8, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=dcf04ca1-f449-4868-acab-73152c97c4c8,segment=NetworkSegment(0d9a1ad5-c1fc-43c7-8293-6032c9456478),segment_id=0d9a1ad5-c1fc-43c7-8293-6032c9456478)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:27:23.848568 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:23.850365 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:23.926758 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=dcf04ca1-f449-4868-acab-73152c97c4c8,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:23.929663 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.079s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:24.321518 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['dcf04ca1-f449-4868-acab-73152c97c4c8'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:27:24.347693 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['04700561-9728-48ff-aa15-d99ae8180704'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:27:24.370694 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6c037789-3e62-4037-8168-a295c29d662b req-6e0912f7-905a-4e80-b1bc-861206309fb2 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:24.372260 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6c037789-3e62-4037-8168-a295c29d662b req-6e0912f7-905a-4e80-b1bc-861206309fb2 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 944 time: 0.1068413 May 20 04:27:24.382564 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f0759523-d402-40dd-a94b-d1e725622618 req-33dba855-a3ef-4fdf-842e-ff7e19303afa tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:24.384027 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f0759523-d402-40dd-a94b-d1e725622618 req-33dba855-a3ef-4fdf-842e-ff7e19303afa tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 949 time: 0.1544466 May 20 04:27:24.454097 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6c037789-3e62-4037-8168-a295c29d662b req-ede27f70-09a0-49af-9aa3-b88029cde48b tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0691054 May 20 04:27:24.457050 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f0759523-d402-40dd-a94b-d1e725622618 req-10e66ba1-2afd-48cb-8366-c20a85c7ff2c tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/security-groups?id=1b091fdd-0a0f-46fb-8de3-aea8fa6fdde8&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0583677 May 20 04:27:24.724097 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6ca4159c-d65e-4469-8d4d-f7fb8b9991e4 req-82fb3ac3-2e36-4567-9a26-751952531b42 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:24.725593 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6ca4159c-d65e-4469-8d4d-f7fb8b9991e4 req-82fb3ac3-2e36-4567-9a26-751952531b42 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 943 time: 0.1733377 May 20 04:27:24.769610 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6ca4159c-d65e-4469-8d4d-f7fb8b9991e4 req-3f0deafd-534c-4b05-9dc5-8008f4732031 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0382624 May 20 04:27:25.287025 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 535d5637-ae66-43cb-bd16-3543bd3b7a8b details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:27:25.423558 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 535d5637-ae66-43cb-bd16-3543bd3b7a8b, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=535d5637-ae66-43cb-bd16-3543bd3b7a8b,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:27:25.437250 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': '535d5637-ae66-43cb-bd16-3543bd3b7a8b', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'port_id': '535d5637-ae66-43cb-bd16-3543bd3b7a8b', 'mac_address': 'fa:16:3e:ea:90:d5', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 847, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.6'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:27:25.497259 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:25.514500 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 535d5637-ae66-43cb-bd16-3543bd3b7a8b {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:27:25.530683 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 535d5637-ae66-43cb-bd16-3543bd3b7a8b, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=535d5637-ae66-43cb-bd16-3543bd3b7a8b,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:27:25.535719 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:25.538478 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:25.611237 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=535d5637-ae66-43cb-bd16-3543bd3b7a8b,revision_number=9']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:25.614765 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.076s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:25.709660 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3a2d2e87-4b1d-479d-b694-85b4e18f75de req-acf3a5bf-72ce-4650-b298-09df7d5c7365 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:25.711019 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3a2d2e87-4b1d-479d-b694-85b4e18f75de req-acf3a5bf-72ce-4650-b298-09df7d5c7365 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 944 time: 0.1470895 May 20 04:27:25.724046 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-24357dea-9c35-45bc-9fa8-c4a814b504f3 req-4861831e-86ee-40a1-afbb-dbe245762a48 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:25.725439 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-24357dea-9c35-45bc-9fa8-c4a814b504f3 req-4861831e-86ee-40a1-afbb-dbe245762a48 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 949 time: 0.1290028 May 20 04:27:25.799330 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3a2d2e87-4b1d-479d-b694-85b4e18f75de req-d7635112-c586-4314-ad16-6f4236d0bd94 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0729318 May 20 04:27:25.801957 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-24357dea-9c35-45bc-9fa8-c4a814b504f3 req-fee0d2c2-c22f-43a5-95f4-250f19bf95c2 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/security-groups?id=1b091fdd-0a0f-46fb-8de3-aea8fa6fdde8&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0631320 May 20 04:27:25.903529 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['535d5637-ae66-43cb-bd16-3543bd3b7a8b'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:27:25.972929 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f45147b3-2c82-4210-b33b-9755b5ff57c5 req-22a4db9e-b068-475d-a77f-d68009ca566e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:25.974455 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f45147b3-2c82-4210-b33b-9755b5ff57c5 req-22a4db9e-b068-475d-a77f-d68009ca566e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 944 time: 0.0802805 May 20 04:27:26.020508 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f45147b3-2c82-4210-b33b-9755b5ff57c5 req-1f597b29-151a-4e4d-8b6e-b41da2b50620 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0398419 May 20 04:27:26.206972 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-524b7cab-2667-4df9-ae0b-664fbdc083e2 req-ac5b613e-aae9-4605-8bed-dcb32b19d00c tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:26.208963 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-524b7cab-2667-4df9-ae0b-664fbdc083e2 req-ac5b613e-aae9-4605-8bed-dcb32b19d00c tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0881345 May 20 04:27:26.254949 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-524b7cab-2667-4df9-ae0b-664fbdc083e2 req-48b72f7c-bb8e-43d8-9e34-d6bedfef9a51 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0399134 May 20 04:27:27.055874 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cfb8f1d7-344d-4491-9402-c225ba69cbb1 req-c63c18ee-d3c3-49d2-9204-61926d3cc846 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:27.057315 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cfb8f1d7-344d-4491-9402-c225ba69cbb1 req-c63c18ee-d3c3-49d2-9204-61926d3cc846 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 944 time: 0.1412237 May 20 04:27:27.064416 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5d3a90aa-2748-4913-b6db-35eaec2eb1ab req-ff9ce9d1-152a-4cc6-81c2-3ed2ac660e38 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:27.065760 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5d3a90aa-2748-4913-b6db-35eaec2eb1ab req-ff9ce9d1-152a-4cc6-81c2-3ed2ac660e38 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 949 time: 0.1191356 May 20 04:27:27.134319 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cfb8f1d7-344d-4491-9402-c225ba69cbb1 req-5d3c3894-3736-49dd-b27c-6a082202bda7 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0676346 May 20 04:27:27.138643 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5d3a90aa-2748-4913-b6db-35eaec2eb1ab req-d60889dc-b1f2-4b1d-bbff-d4adac240301 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/security-groups?id=1b091fdd-0a0f-46fb-8de3-aea8fa6fdde8&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0593159 May 20 04:27:27.306164 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 535d5637-ae66-43cb-bd16-3543bd3b7a8b up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:27:27.313622 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7a186526-0f64-4c11-b309-009e61315ac9 req-e16a8b71-9933-43e8-a788-bced2bcc7677 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:27.315256 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7a186526-0f64-4c11-b309-009e61315ac9 req-e16a8b71-9933-43e8-a788-bced2bcc7677 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 944 time: 0.0875740 May 20 04:27:27.362325 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7a186526-0f64-4c11-b309-009e61315ac9 req-66024c3c-48bb-41e1-a75e-859771c6d61b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0408320 May 20 04:27:27.376431 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port 535d5637-ae66-43cb-bd16-3543bd3b7a8b triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:27:27.376817 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:27.456560 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8a44e9c1-17d0-4e4a-9dc0-571f58058e22 req-71cb5f52-3b39-48fb-8ad7-d34555dc383b tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:27.457990 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8a44e9c1-17d0-4e4a-9dc0-571f58058e22 req-71cb5f52-3b39-48fb-8ad7-d34555dc383b tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0832093 May 20 04:27:27.491840 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:27.502717 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8a44e9c1-17d0-4e4a-9dc0-571f58058e22 req-c0a80a0f-e946-48c8-ae07-da1c36ea6a5c tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0387020 May 20 04:27:27.518369 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'c2ca950f-78ff-4e44-9231-e8e19f4aee5d', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '535d5637-ae66-43cb-bd16-3543bd3b7a8b'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:27:27.528276 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 535d5637-ae66-43cb-bd16-3543bd3b7a8b, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=535d5637-ae66-43cb-bd16-3543bd3b7a8b,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:27:27.533842 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:27.534687 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 04700561-9728-48ff-aa15-d99ae8180704 up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:27:27.562287 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:27.593718 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'c2ca950f-78ff-4e44-9231-e8e19f4aee5d', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '535d5637-ae66-43cb-bd16-3543bd3b7a8b', 'code': 200} May 20 04:27:27.650243 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=535d5637-ae66-43cb-bd16-3543bd3b7a8b,revision_number=10']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:27.653524 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.091s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:27.663444 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port 04700561-9728-48ff-aa15-d99ae8180704 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:27:27.668238 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port 04700561-9728-48ff-aa15-d99ae8180704 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:27:27.668617 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:27.786688 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:27.820780 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 04700561-9728-48ff-aa15-d99ae8180704, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=04700561-9728-48ff-aa15-d99ae8180704,segment=NetworkSegment(0fbdfc2a-8204-4d65-bf3c-9044da0482d3),segment_id=0fbdfc2a-8204-4d65-bf3c-9044da0482d3)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:27:27.826566 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:27.827556 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device dcf04ca1-f449-4868-acab-73152c97c4c8 up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:27:27.851515 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-dfc6f6f8-a39a-4ab9-a478-4083e76fe6c7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:27.941493 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=04700561-9728-48ff-aa15-d99ae8180704,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:27.944538 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.093s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:27.956574 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port dcf04ca1-f449-4868-acab-73152c97c4c8 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:27:27.961530 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port dcf04ca1-f449-4868-acab-73152c97c4c8 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:27:27.961905 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:28.081956 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:28.117259 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port dcf04ca1-f449-4868-acab-73152c97c4c8, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=dcf04ca1-f449-4868-acab-73152c97c4c8,segment=NetworkSegment(0d9a1ad5-c1fc-43c7-8293-6032c9456478),segment_id=0d9a1ad5-c1fc-43c7-8293-6032c9456478)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:27:28.123161 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:28.125199 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:28.190703 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=dcf04ca1-f449-4868-acab-73152c97c4c8,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:28.193984 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:28.344834 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0190fe59-3114-40c1-ba94-636e5e9957bd req-a5d0cbe0-50d9-4f92-8308-62959651be03 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:28.346254 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0190fe59-3114-40c1-ba94-636e5e9957bd req-a5d0cbe0-50d9-4f92-8308-62959651be03 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 1099 time: 0.1624479 May 20 04:27:28.413048 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0190fe59-3114-40c1-ba94-636e5e9957bd req-9560051b-db36-48ce-b7d4-b87cc536ac73 service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0583754 May 20 04:27:28.434685 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-479a2793-f536-41bf-97ee-2546b6e36a14 req-44506523-0b8e-49f6-a96f-00327b40f692 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:28.436074 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-479a2793-f536-41bf-97ee-2546b6e36a14 req-44506523-0b8e-49f6-a96f-00327b40f692 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 950 time: 0.1457670 May 20 04:27:28.443236 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2ca8bffd-f5b9-4d7c-8ea2-eb5ad941a528 req-c6af8552-f10c-40c2-bbed-68652eec0733 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:28.444625 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2ca8bffd-f5b9-4d7c-8ea2-eb5ad941a528 req-c6af8552-f10c-40c2-bbed-68652eec0733 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 945 time: 0.1933501 May 20 04:27:28.510910 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-479a2793-f536-41bf-97ee-2546b6e36a14 req-467f032f-0d61-48b8-8bfe-5c973828d49a tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/security-groups?id=1b091fdd-0a0f-46fb-8de3-aea8fa6fdde8&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0654380 May 20 04:27:28.515306 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2ca8bffd-f5b9-4d7c-8ea2-eb5ad941a528 req-40b40774-5385-4839-be11-c8c88af221a8 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0573170 May 20 04:27:28.750468 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-271c2340-46c8-4a31-8c7a-b88410d11311 req-1a8f8c6d-51cd-4e8a-8436-a242e4776cd1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:28.751837 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-271c2340-46c8-4a31-8c7a-b88410d11311 req-1a8f8c6d-51cd-4e8a-8436-a242e4776cd1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 946 time: 0.1424687 May 20 04:27:28.761689 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-83cdebcf-0355-4b54-b156-e888b4ac299f req-655106bf-08c1-4de9-96e5-9671a0eceb42 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:28.763002 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-83cdebcf-0355-4b54-b156-e888b4ac299f req-655106bf-08c1-4de9-96e5-9671a0eceb42 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.1519923 May 20 04:27:28.827521 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-271c2340-46c8-4a31-8c7a-b88410d11311 req-dd1c30af-f151-4159-a293-f00f2f82c4f3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0635531 May 20 04:27:28.834830 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-83cdebcf-0355-4b54-b156-e888b4ac299f req-6a49ad1c-064f-4835-8fa6-7ee5fd20772e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0580320 May 20 04:27:29.594788 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '259bb3a0-b99c-43ed-9876-ef59563646c3', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '04700561-9728-48ff-aa15-d99ae8180704'}, {'server_uuid': '5a913a0a-8f85-45b8-a226-396c3539d204', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'dcf04ca1-f449-4868-acab-73152c97c4c8'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:27:29.751907 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d79df7c-1c6a-4921-a8ff-53f31a97f47f req-0f1acea5-adb6-4500-a870-0cdeec664e4c tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:29.753734 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1d79df7c-1c6a-4921-a8ff-53f31a97f47f req-0f1acea5-adb6-4500-a870-0cdeec664e4c tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 950 time: 0.0932519 May 20 04:27:29.804761 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1d79df7c-1c6a-4921-a8ff-53f31a97f47f req-a12b7631-4b16-4b4a-9102-cacee9927369 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/security-groups?id=1b091fdd-0a0f-46fb-8de3-aea8fa6fdde8&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0444567 May 20 04:27:29.865033 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '259bb3a0-b99c-43ed-9876-ef59563646c3', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '04700561-9728-48ff-aa15-d99ae8180704', 'code': 200} May 20 04:27:29.865303 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '5a913a0a-8f85-45b8-a226-396c3539d204', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'dcf04ca1-f449-4868-acab-73152c97c4c8', 'code': 200} May 20 04:27:29.909542 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a61ed4ec-43c8-43b5-85ae-9ac13b2642ef req-73963313-ed6e-4dba-a573-481baa499aa5 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:29.911857 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a61ed4ec-43c8-43b5-85ae-9ac13b2642ef req-73963313-ed6e-4dba-a573-481baa499aa5 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 945 time: 0.1026130 May 20 04:27:30.139821 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a61ed4ec-43c8-43b5-85ae-9ac13b2642ef req-4d43854e-f4f4-411f-8211-13018b91153b tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2195997 May 20 04:27:30.232756 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-07f8f137-8724-483e-b06f-96ac373049fd req-0195cb32-def7-4067-9c45-c5526fcca26d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:30.234508 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-07f8f137-8724-483e-b06f-96ac373049fd req-0195cb32-def7-4067-9c45-c5526fcca26d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 946 time: 0.2821453 May 20 04:27:30.246443 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-08dde932-c26c-47b2-94e7-ac8b0aa990e0 req-e1db1674-e417-493f-9c1d-7f813b67dc65 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:30.248402 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-08dde932-c26c-47b2-94e7-ac8b0aa990e0 req-e1db1674-e417-493f-9c1d-7f813b67dc65 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 1103 time: 0.0990937 May 20 04:27:30.256033 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c773c1ad-d692-4984-b84f-ecd883f0ffa0 req-2a4b246c-7a21-48a7-a162-6c73489e82de service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:30.257567 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c773c1ad-d692-4984-b84f-ecd883f0ffa0 req-2a4b246c-7a21-48a7-a162-6c73489e82de service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 1098 time: 0.1080117 May 20 04:27:30.311101 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c773c1ad-d692-4984-b84f-ecd883f0ffa0 req-7d68b28e-f84c-4c3c-af3f-2705dd8787de service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0373368 May 20 04:27:30.345928 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-07f8f137-8724-483e-b06f-96ac373049fd req-83190568-d89e-4e1c-98af-eae63251658c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0960629 May 20 04:27:30.375629 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-08dde932-c26c-47b2-94e7-ac8b0aa990e0 req-d4bc6c3e-f41d-4377-9510-f664fe7b30ab service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=1b091fdd-0a0f-46fb-8de3-aea8fa6fdde8&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0952713 May 20 04:27:30.411939 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b33f06a9-d810-4590-ac6f-aebeb8d9aa54 req-24967ded-ded5-4eb7-a2c5-f7c2c1e16119 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:30.413392 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b33f06a9-d810-4590-ac6f-aebeb8d9aa54 req-24967ded-ded5-4eb7-a2c5-f7c2c1e16119 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.1166306 May 20 04:27:30.457914 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b33f06a9-d810-4590-ac6f-aebeb8d9aa54 req-cfcfe1e0-c98c-4691-8c6a-377a8c74644d tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0387170 May 20 04:27:31.015050 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-511df04f-cf60-4dca-8b3b-ecc05b2423e6 req-3fc2fb11-799f-46c4-b398-3a5928b60428 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:31.016517 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-511df04f-cf60-4dca-8b3b-ecc05b2423e6 req-3fc2fb11-799f-46c4-b398-3a5928b60428 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 950 time: 0.0909255 May 20 04:27:31.062724 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-511df04f-cf60-4dca-8b3b-ecc05b2423e6 req-60da9161-4c2e-4fc2-8a77-b05da0292cb1 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/security-groups?id=1b091fdd-0a0f-46fb-8de3-aea8fa6fdde8&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0403128 May 20 04:27:31.374516 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-032bf266-44ca-4e0e-b6d3-e49aeb29862d req-af676b30-685e-402d-937e-be1ea4a48274 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:31.376327 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-032bf266-44ca-4e0e-b6d3-e49aeb29862d req-af676b30-685e-402d-937e-be1ea4a48274 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 945 time: 0.1005905 May 20 04:27:31.420356 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-032bf266-44ca-4e0e-b6d3-e49aeb29862d req-48aaac18-f5bb-463d-993c-95d42e1f6348 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0385807 May 20 04:27:31.541356 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-983a8d7e-db65-45e4-84d9-9ca2d7b56b70 req-5cdf22de-a9c9-4ba7-ad9d-4d3bb3d1ed33 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:31.543107 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-983a8d7e-db65-45e4-84d9-9ca2d7b56b70 req-5cdf22de-a9c9-4ba7-ad9d-4d3bb3d1ed33 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 946 time: 0.0855362 May 20 04:27:31.633016 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-983a8d7e-db65-45e4-84d9-9ca2d7b56b70 req-a6db3430-e3b3-4ba9-a152-c18f5a999d59 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0839496 May 20 04:27:31.664792 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f2134c42-7571-4b92-8a5b-939004bd038a req-5b546c51-9a44-4fb4-ab02-afeb500a04c2 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:31.666225 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f2134c42-7571-4b92-8a5b-939004bd038a req-5b546c51-9a44-4fb4-ab02-afeb500a04c2 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0994422 May 20 04:27:31.711237 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f2134c42-7571-4b92-8a5b-939004bd038a req-ed3d35dd-159f-40bc-9baf-451e63dfcb53 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0391333 May 20 04:27:32.255132 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d1957f42-72a6-42b4-94d3-98121553078d req-2de7237f-d96a-4be1-9ea2-724d08cb22fc tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:32.256618 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d1957f42-72a6-42b4-94d3-98121553078d req-2de7237f-d96a-4be1-9ea2-724d08cb22fc tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 950 time: 0.0868015 May 20 04:27:32.302235 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d1957f42-72a6-42b4-94d3-98121553078d req-570eb2e3-fe24-4d2d-b537-8edb289e1f6c tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/security-groups?id=1b091fdd-0a0f-46fb-8de3-aea8fa6fdde8&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0397639 May 20 04:27:32.612715 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e917fdc4-5275-42aa-b40f-df9ebade66f1 req-c595c696-9054-4c2c-8bf6-7af8a3effd36 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:32.614135 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e917fdc4-5275-42aa-b40f-df9ebade66f1 req-c595c696-9054-4c2c-8bf6-7af8a3effd36 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 945 time: 0.0898113 May 20 04:27:32.662639 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e917fdc4-5275-42aa-b40f-df9ebade66f1 req-2453c4b8-e2fd-45de-ba62-7513e7987cf6 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0427361 May 20 04:27:32.901798 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cfa5f2de-e5de-4a89-b8ce-a1079a473f4a req-7d60da3f-47b2-4854-aa89-ccba781f5e2c tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:32.903494 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cfa5f2de-e5de-4a89-b8ce-a1079a473f4a req-7d60da3f-47b2-4854-aa89-ccba781f5e2c tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0844517 May 20 04:27:32.948573 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cfa5f2de-e5de-4a89-b8ce-a1079a473f4a req-8e022a1a-bee3-4d41-b892-b44db8c74619 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0378315 May 20 04:27:33.493308 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c4c0ca98-92cc-4c79-956a-21c7729e1098 req-3cba2d90-ec70-412c-bfbd-c5e237b956a3 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:33.494747 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c4c0ca98-92cc-4c79-956a-21c7729e1098 req-3cba2d90-ec70-412c-bfbd-c5e237b956a3 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 950 time: 0.0835183 May 20 04:27:33.539196 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c4c0ca98-92cc-4c79-956a-21c7729e1098 req-5e38df40-75f3-4f02-b8e6-b375803dbe73 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/security-groups?id=1b091fdd-0a0f-46fb-8de3-aea8fa6fdde8&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0385575 May 20 04:27:33.838361 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-df6a6586-e675-46a0-94c3-443d4f26a60b req-3dda151c-3d1d-4049-bd22-3759cf01ff09 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:33.839850 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-df6a6586-e675-46a0-94c3-443d4f26a60b req-3dda151c-3d1d-4049-bd22-3759cf01ff09 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 945 time: 0.0849915 May 20 04:27:33.887187 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-df6a6586-e675-46a0-94c3-443d4f26a60b req-72fbcba8-c414-4ec5-b421-fbc359b080b4 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0409567 May 20 04:27:33.980022 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f5c52d8a-684a-4ebc-b0b4-a732848e1b78 req-0cca4034-6023-4989-9e03-e41452562ca8 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:33.981461 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f5c52d8a-684a-4ebc-b0b4-a732848e1b78 req-0cca4034-6023-4989-9e03-e41452562ca8 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 950 time: 0.0805726 May 20 04:27:34.026871 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f5c52d8a-684a-4ebc-b0b4-a732848e1b78 req-5ec1efb3-0c62-42e1-87c7-b4d23773c572 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/security-groups?id=1b091fdd-0a0f-46fb-8de3-aea8fa6fdde8&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0396838 May 20 04:27:34.148067 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cbb459b6-3f0a-4ecb-867d-e8eb3bb98d5a req-4f35f47c-af30-4483-ae39-64bbf184bb17 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:34.149412 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cbb459b6-3f0a-4ecb-867d-e8eb3bb98d5a req-4f35f47c-af30-4483-ae39-64bbf184bb17 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0842505 May 20 04:27:34.197886 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cbb459b6-3f0a-4ecb-867d-e8eb3bb98d5a req-343becbf-a2b3-4adf-86f6-aea72d50bf1a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0399816 May 20 04:27:35.081567 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4fcfceb4-4e9a-4d89-8efa-09f1df645385 req-66a49831-c3c3-45de-a998-39b7955df3d3 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:35.083008 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4fcfceb4-4e9a-4d89-8efa-09f1df645385 req-66a49831-c3c3-45de-a998-39b7955df3d3 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 945 time: 0.0828364 May 20 04:27:35.132068 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4fcfceb4-4e9a-4d89-8efa-09f1df645385 req-31f55fe0-59d0-4377-8e11-e1cd7f712b7f tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0430744 May 20 04:27:35.211048 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e9ee61c-af26-493e-8118-feca32fc3347 req-6184562f-06f2-44b7-83d6-bf73a62835ad tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:35.212430 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8e9ee61c-af26-493e-8118-feca32fc3347 req-6184562f-06f2-44b7-83d6-bf73a62835ad tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 950 time: 0.0794597 May 20 04:27:35.257916 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8e9ee61c-af26-493e-8118-feca32fc3347 req-c63095ce-3174-4004-9248-b778ce97252c tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/security-groups?id=1b091fdd-0a0f-46fb-8de3-aea8fa6fdde8&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0393419 May 20 04:27:35.474454 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dd591c01-3fa4-4a71-a1b1-b2698e4f5364 req-3f7e479d-2121-46e8-8de6-8bdab6097ab1 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:35.475912 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-dd591c01-3fa4-4a71-a1b1-b2698e4f5364 req-3f7e479d-2121-46e8-8de6-8bdab6097ab1 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.1703315 May 20 04:27:35.520870 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-dd591c01-3fa4-4a71-a1b1-b2698e4f5364 req-75b8696f-fa10-48a8-a60f-5d7eeefa8e34 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0392015 May 20 04:27:35.586909 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:35.693126 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-671c60f5-11b8-4c8d-a12e-1fd2b79584a8 req-79902e6b-96f4-4b0a-bf68-ca33f9a4066f tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:35.694538 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-671c60f5-11b8-4c8d-a12e-1fd2b79584a8 req-79902e6b-96f4-4b0a-bf68-ca33f9a4066f tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.74 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 945 time: 0.0827696 May 20 04:27:35.838333 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-671c60f5-11b8-4c8d-a12e-1fd2b79584a8 req-8d107e4f-1057-4305-b09e-179c4265f0d7 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Deleting port dcf04ca1-f449-4868-acab-73152c97c4c8 {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:27:35.838750 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-671c60f5-11b8-4c8d-a12e-1fd2b79584a8 req-8d107e4f-1057-4305-b09e-179c4265f0d7 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:35.979148 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-172537aa-e87b-4b63-afa1-ac614776bc62 req-686396f9-cb01-40d3-8372-14e0a83a8efe tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:35.981434 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-172537aa-e87b-4b63-afa1-ac614776bc62 req-686396f9-cb01-40d3-8372-14e0a83a8efe tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.74 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 950 time: 0.0859797 May 20 04:27:35.985676 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-671c60f5-11b8-4c8d-a12e-1fd2b79584a8 req-8d107e4f-1057-4305-b09e-179c4265f0d7 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] For port dcf04ca1-f449-4868-acab-73152c97c4c8, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=dcf04ca1-f449-4868-acab-73152c97c4c8,segment=NetworkSegment(0d9a1ad5-c1fc-43c7-8293-6032c9456478),segment_id=0d9a1ad5-c1fc-43c7-8293-6032c9456478)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:27:35.986040 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-671c60f5-11b8-4c8d-a12e-1fd2b79584a8 req-8d107e4f-1057-4305-b09e-179c4265f0d7 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:36.000011 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:27:36.110391 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-671c60f5-11b8-4c8d-a12e-1fd2b79584a8 req-8d107e4f-1057-4305-b09e-179c4265f0d7 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Calling delete_port for dcf04ca1-f449-4868-acab-73152c97c4c8 owned by compute:nova {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:27:36.125911 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [req-172537aa-e87b-4b63-afa1-ac614776bc62 req-44ea8d0e-a80a-456c-b4e0-ec66a5fae0af tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Deleting port 04700561-9728-48ff-aa15-d99ae8180704 {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:27:36.126262 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-172537aa-e87b-4b63-afa1-ac614776bc62 req-44ea8d0e-a80a-456c-b4e0-ec66a5fae0af tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:36.231012 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-671c60f5-11b8-4c8d-a12e-1fd2b79584a8 req-8d107e4f-1057-4305-b09e-179c4265f0d7 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:36.243120 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:36.244901 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['dcf04ca1-f449-4868-acab-73152c97c4c8'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:27:36.263047 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-172537aa-e87b-4b63-afa1-ac614776bc62 req-44ea8d0e-a80a-456c-b4e0-ec66a5fae0af tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] For port 04700561-9728-48ff-aa15-d99ae8180704, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=04700561-9728-48ff-aa15-d99ae8180704,segment=NetworkSegment(0fbdfc2a-8204-4d65-bf3c-9044da0482d3),segment_id=0fbdfc2a-8204-4d65-bf3c-9044da0482d3)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:27:36.263435 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-172537aa-e87b-4b63-afa1-ac614776bc62 req-44ea8d0e-a80a-456c-b4e0-ec66a5fae0af tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:36.352700 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8d107e4f-1057-4305-b09e-179c4265f0d7 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Pushing event deleted for resources: {'Port': ['ID=dcf04ca1-f449-4868-acab-73152c97c4c8,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:36.361826 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-8d107e4f-1057-4305-b09e-179c4265f0d7 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.119s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:36.364295 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d5e6543a-6165-4494-a36b-fca1b9461c90 req-1bee7c5b-242d-4312-9c8f-348eef6d96ef tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=5a913a0a-8f85-45b8-a226-396c3539d204 HTTP/1.1" status: 200 len: 210 time: 0.0868618 May 20 04:27:36.388354 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [req-172537aa-e87b-4b63-afa1-ac614776bc62 req-44ea8d0e-a80a-456c-b4e0-ec66a5fae0af tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Calling delete_port for 04700561-9728-48ff-aa15-d99ae8180704 owned by compute:nova {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:27:36.431860 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [req-671c60f5-11b8-4c8d-a12e-1fd2b79584a8 req-8d107e4f-1057-4305-b09e-179c4265f0d7 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] No DVR routers for this DVR port dcf04ca1-f449-4868-acab-73152c97c4c8 on host hv-cinder-78240 {{(pid=91643) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:27:36.438229 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-671c60f5-11b8-4c8d-a12e-1fd2b79584a8 req-8d107e4f-1057-4305-b09e-179c4265f0d7 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:36.440826 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '5a913a0a-8f85-45b8-a226-396c3539d204', 'name': 'network-vif-deleted', 'tag': 'dcf04ca1-f449-4868-acab-73152c97c4c8'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:27:36.457079 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-671c60f5-11b8-4c8d-a12e-1fd2b79584a8 req-8d107e4f-1057-4305-b09e-179c4265f0d7 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.74 "DELETE /v2.0/ports/dcf04ca1-f449-4868-acab-73152c97c4c8 HTTP/1.1" status: 204 len: 173 time: 0.7565861 May 20 04:27:36.497359 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6873fb73-ae6d-42cd-9fa5-80193995fdbd req-21901374-db36-4307-ba83-3800e6272599 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:36.499490 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6873fb73-ae6d-42cd-9fa5-80193995fdbd req-21901374-db36-4307-ba83-3800e6272599 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/ports?device_id=259bb3a0-b99c-43ed-9876-ef59563646c3 HTTP/1.1" status: 200 len: 950 time: 0.1051118 May 20 04:27:36.503124 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-172537aa-e87b-4b63-afa1-ac614776bc62 req-44ea8d0e-a80a-456c-b4e0-ec66a5fae0af tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:36.512218 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-5820cc2c-3cdd-49e4-a270-24544662a402 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:36.515933 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['04700561-9728-48ff-aa15-d99ae8180704'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:27:36.521061 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '5a913a0a-8f85-45b8-a226-396c3539d204', 'name': 'network-vif-deleted', 'tag': 'dcf04ca1-f449-4868-acab-73152c97c4c8', 'status': 'completed', 'code': 200} May 20 04:27:36.564101 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6873fb73-ae6d-42cd-9fa5-80193995fdbd req-a6735674-b862-4718-b786-4898dd04a9b4 tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.39 "GET /v2.0/security-groups?id=1b091fdd-0a0f-46fb-8de3-aea8fa6fdde8&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0578449 May 20 04:27:36.567547 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-44ea8d0e-a80a-456c-b4e0-ec66a5fae0af tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Pushing event deleted for resources: {'Port': ['ID=04700561-9728-48ff-aa15-d99ae8180704,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:36.571761 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-44ea8d0e-a80a-456c-b4e0-ec66a5fae0af tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.060s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:36.658364 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-172537aa-e87b-4b63-afa1-ac614776bc62 req-44ea8d0e-a80a-456c-b4e0-ec66a5fae0af tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] No DVR routers for this DVR port 04700561-9728-48ff-aa15-d99ae8180704 on host hv-cinder-78240 {{(pid=91644) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:27:36.661090 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-172537aa-e87b-4b63-afa1-ac614776bc62 req-44ea8d0e-a80a-456c-b4e0-ec66a5fae0af tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:36.665195 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '259bb3a0-b99c-43ed-9876-ef59563646c3', 'name': 'network-vif-deleted', 'tag': '04700561-9728-48ff-aa15-d99ae8180704'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:27:36.681950 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-172537aa-e87b-4b63-afa1-ac614776bc62 req-44ea8d0e-a80a-456c-b4e0-ec66a5fae0af tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] 10.222.0.74 "DELETE /v2.0/ports/04700561-9728-48ff-aa15-d99ae8180704 HTTP/1.1" status: 204 len: 173 time: 0.6937580 May 20 04:27:36.719545 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '259bb3a0-b99c-43ed-9876-ef59563646c3', 'name': 'network-vif-deleted', 'tag': '04700561-9728-48ff-aa15-d99ae8180704', 'status': 'completed', 'code': 200} May 20 04:27:36.777389 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2ec9121e-93f7-4730-8199-5c465b01b73d req-f84e8418-4a6b-499a-8ed9-0d5f71191892 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:36.779376 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2ec9121e-93f7-4730-8199-5c465b01b73d req-f84e8418-4a6b-499a-8ed9-0d5f71191892 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.1048479 May 20 04:27:36.828010 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2ec9121e-93f7-4730-8199-5c465b01b73d req-e2aeab88-2aae-4f35-b337-6c115e670d10 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0428102 May 20 04:27:37.788172 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-2d1c02dd-8d99-4e9d-a7cb-7221fb80ed4b tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:37.789608 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-2d1c02dd-8d99-4e9d-a7cb-7221fb80ed4b tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/networks?id=e3248a7c-d6dc-404f-9bb7-8e87d070d1cb HTTP/1.1" status: 200 len: 808 time: 0.1487162 May 20 04:27:37.804578 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-6de02a55-2d6b-4188-974c-b8b7b6c9f077 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/quotas/c57d56efd5fb40069159f50973655a5d HTTP/1.1" status: 200 len: 375 time: 0.0093150 May 20 04:27:37.853150 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-0f7b1762-b3e1-4ba8-8f49-ef83bf9e3a9d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Synchronizing usage tracker for tenant:c57d56efd5fb40069159f50973655a5d on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:27:37.870568 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-0f7b1762-b3e1-4ba8-8f49-ef83bf9e3a9d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Unset dirty status for tenant:c57d56efd5fb40069159f50973655a5d on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:27:37.871925 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-0f7b1762-b3e1-4ba8-8f49-ef83bf9e3a9d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=c57d56efd5fb40069159f50973655a5d&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0620544 May 20 04:27:38.060263 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-76b911b5-2e15-496d-a457-81471badd17c service neutron] 10.222.0.39 "GET /v2.0/networks/e3248a7c-d6dc-404f-9bb7-8e87d070d1cb?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1804540 May 20 04:27:38.120347 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d240b59a-1823-42b5-960a-0c8379faf5bc req-5bbedb63-e9dc-46e5-b572-8d83015494c2 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:38.121827 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d240b59a-1823-42b5-960a-0c8379faf5bc req-5bbedb63-e9dc-46e5-b572-8d83015494c2 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.1555867 May 20 04:27:38.336837 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d240b59a-1823-42b5-960a-0c8379faf5bc req-8e438910-a439-4ad5-a337-867d9e03504b tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1705575 May 20 04:27:38.342573 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-53d8f11d-3809-46f8-96a6-b4013bf0e6de service neutron] 10.222.0.39 "GET /v2.0/networks/e3248a7c-d6dc-404f-9bb7-8e87d070d1cb?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2740335 May 20 04:27:38.521536 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f6b6be44-6866-4cc8-99b8-eec00cddd01a req-22030a28-1969-482a-aab0-8235be784b0d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfb7dc32-7bb4-4030-890b-55321eae3099 HTTP/1.1" status: 200 len: 210 time: 0.0440538 May 20 04:27:39.333690 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 04700561-9728-48ff-aa15-d99ae8180704 no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:27:39.370973 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: 04700561-9728-48ff-aa15-d99ae8180704 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:27:39.372001 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 04700561-9728-48ff-aa15-d99ae8180704 not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:27:39.372001 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device dcf04ca1-f449-4868-acab-73152c97c4c8 no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:27:39.411800 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: dcf04ca1-f449-4868-acab-73152c97c4c8 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:27:39.412436 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device dcf04ca1-f449-4868-acab-73152c97c4c8 not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:27:39.554606 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bdf7b9d4-1541-4d0d-8c23-a2ac201736f3 req-07b289d5-46c8-43e5-891d-55e95a5335d9 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:39.563970 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bdf7b9d4-1541-4d0d-8c23-a2ac201736f3 req-07b289d5-46c8-43e5-891d-55e95a5335d9 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.1035867 May 20 04:27:39.614878 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bdf7b9d4-1541-4d0d-8c23-a2ac201736f3 req-f0eb127c-56a3-4293-8969-7b5cd78508a0 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0498807 May 20 04:27:39.621177 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-a3fc7dc3-4ba0-4ea4-8d8a-9c9722162661 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:39.623255 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-a3fc7dc3-4ba0-4ea4-8d8a-9c9722162661 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.74 "GET /v2.0/networks?id=e3248a7c-d6dc-404f-9bb7-8e87d070d1cb HTTP/1.1" status: 200 len: 808 time: 0.1652639 May 20 04:27:39.641842 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-0e45daa8-d372-4095-893b-4904036cdd7d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Request body: {'port': {'device_id': 'dfb7dc32-7bb4-4030-890b-55321eae3099', 'network_id': 'e3248a7c-d6dc-404f-9bb7-8e87d070d1cb', 'admin_state_up': True, 'tenant_id': 'c57d56efd5fb40069159f50973655a5d'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:27:39.724724 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9aface99-c68d-488f-9a5e-f154bcd80e3b req-72dc5245-bc77-4456-9be8-40586df11faa tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfb7dc32-7bb4-4030-890b-55321eae3099 HTTP/1.1" status: 200 len: 210 time: 0.0544348 May 20 04:27:39.788140 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-0e45daa8-d372-4095-893b-4904036cdd7d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:27:39.798156 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-0e45daa8-d372-4095-893b-4904036cdd7d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:27:39.808619 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-0e45daa8-d372-4095-893b-4904036cdd7d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Made reservation on behalf of c57d56efd5fb40069159f50973655a5d for: {'port': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:27:39.812498 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-0e45daa8-d372-4095-893b-4904036cdd7d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:40.070193 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-0e45daa8-d372-4095-893b-4904036cdd7d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Allocated IP 172.20.0.3 (e3248a7c-d6dc-404f-9bb7-8e87d070d1cb/cb4fcf1d-e5d6-469d-83a3-5d90ce39e474/95c6e357-0f6b-489a-b926-e20fd94fd9dc) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:27:40.277376 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-0e45daa8-d372-4095-893b-4904036cdd7d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:40.307693 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-0e45daa8-d372-4095-893b-4904036cdd7d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:40.313131 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:40.383387 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-0e45daa8-d372-4095-893b-4904036cdd7d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Network e3248a7c-d6dc-404f-9bb7-8e87d070d1cb is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:27:40.386626 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-0e45daa8-d372-4095-893b-4904036cdd7d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:27:40.390722 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-0e45daa8-d372-4095-893b-4904036cdd7d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:27:40.421025 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0e45daa8-d372-4095-893b-4904036cdd7d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Pushing event updated for resources: {'Port': ['ID=95c6e357-0f6b-489a-b926-e20fd94fd9dc,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:40.427614 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-0e45daa8-d372-4095-893b-4904036cdd7d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:40.457085 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-0e45daa8-d372-4095-893b-4904036cdd7d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:40.457697 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-0e45daa8-d372-4095-893b-4904036cdd7d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:40.483344 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-0e45daa8-d372-4095-893b-4904036cdd7d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 0.8455625 May 20 04:27:40.499055 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Request body: {'port': {'device_id': 'dfb7dc32-7bb4-4030-890b-55321eae3099', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:27:40.608981 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:40.827733 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-660da19c-2f16-4dcb-aa53-731b13c7b0fa req-18f3f5f3-5b7b-414b-9c26-014ec6f42a01 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:40.829884 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-660da19c-2f16-4dcb-aa53-731b13c7b0fa req-18f3f5f3-5b7b-414b-9c26-014ec6f42a01 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0975044 May 20 04:27:40.933590 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-660da19c-2f16-4dcb-aa53-731b13c7b0fa req-324995c9-70de-4443-b01e-1677ec2ba9c0 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0955675 May 20 04:27:40.960677 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8d4a1d11-74d7-4f47-a9f3-0824585f9949 req-a94d1075-6264-486a-b241-b69d876868d4 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:40.962155 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8d4a1d11-74d7-4f47-a9f3-0824585f9949 req-a94d1075-6264-486a-b241-b69d876868d4 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfb7dc32-7bb4-4030-890b-55321eae3099 HTTP/1.1" status: 200 len: 931 time: 0.1097565 May 20 04:27:40.995826 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 95c6e357-0f6b-489a-b926-e20fd94fd9dc {{(pid=91644) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:27:41.004174 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:41.010007 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8d4a1d11-74d7-4f47-a9f3-0824585f9949 req-5e582e06-1015-4615-8394-227b8f926fd2 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0412021 May 20 04:27:41.019421 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Transition to ACTIVE for port object 95c6e357-0f6b-489a-b926-e20fd94fd9dc will not be triggered until provisioned by entity L2. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:27:41.023663 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:41.028295 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-eec55cf8-a8b1-41d3-8903-cb5b3a355bcf admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:41.187583 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Pushing event updated for resources: {'Port': ['ID=95c6e357-0f6b-489a-b926-e20fd94fd9dc,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:41.195112 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.166s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:41.264136 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Attempting to bind port 95c6e357-0f6b-489a-b926-e20fd94fd9dc on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:27:41.264462 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Attempting to bind port 95c6e357-0f6b-489a-b926-e20fd94fd9dc by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': '0d9a1ad5-c1fc-43c7-8293-6032c9456478', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 809, 'network_id': 'e3248a7c-d6dc-404f-9bb7-8e87d070d1cb'}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:27:41.264785 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Attempting to bind port 95c6e357-0f6b-489a-b926-e20fd94fd9dc on network e3248a7c-d6dc-404f-9bb7-8e87d070d1cb {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:27:41.270070 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Port 95c6e357-0f6b-489a-b926-e20fd94fd9dc on network e3248a7c-d6dc-404f-9bb7-8e87d070d1cb not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:27:41.270384 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Attempting to bind port 95c6e357-0f6b-489a-b926-e20fd94fd9dc on network e3248a7c-d6dc-404f-9bb7-8e87d070d1cb {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:27:41.275684 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 27, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:27:41.276018 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Checking segment: {'id': '0d9a1ad5-c1fc-43c7-8293-6032c9456478', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 809, 'network_id': 'e3248a7c-d6dc-404f-9bb7-8e87d070d1cb'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:27:41.276414 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Bound using segment: {'id': '0d9a1ad5-c1fc-43c7-8293-6032c9456478', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 809, 'network_id': 'e3248a7c-d6dc-404f-9bb7-8e87d070d1cb'} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:27:41.276867 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Bound port: 95c6e357-0f6b-489a-b926-e20fd94fd9dc, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '0d9a1ad5-c1fc-43c7-8293-6032c9456478', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 809, 'network_id': 'e3248a7c-d6dc-404f-9bb7-8e87d070d1cb'}}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:27:41.277195 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:41.370832 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] For port 95c6e357-0f6b-489a-b926-e20fd94fd9dc, host hv-cinder-78240, cleared binding levels {{(pid=91644) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:27:41.374484 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] For port 95c6e357-0f6b-489a-b926-e20fd94fd9dc, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=95c6e357-0f6b-489a-b926-e20fd94fd9dc,segment=NetworkSegment(0d9a1ad5-c1fc-43c7-8293-6032c9456478),segment_id=0d9a1ad5-c1fc-43c7-8293-6032c9456478)] {{(pid=91644) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:27:41.382293 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:41.387313 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d99446f4-00e6-4dab-8f98-ef2fd73bd4b9 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:41.451827 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:41.452433 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:41.453848 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] 10.222.0.74 "PUT /v2.0/ports/95c6e357-0f6b-489a-b926-e20fd94fd9dc HTTP/1.1" status: 200 len: 1093 time: 0.9625349 May 20 04:27:41.454316 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'dfb7dc32-7bb4-4030-890b-55321eae3099', 'tag': '95c6e357-0f6b-489a-b926-e20fd94fd9dc'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:27:41.476440 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Pushing event updated for resources: {'Port': ['ID=95c6e357-0f6b-489a-b926-e20fd94fd9dc,revision_number=3']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:41.479474 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.093s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:41.510192 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'dfb7dc32-7bb4-4030-890b-55321eae3099', 'tag': '95c6e357-0f6b-489a-b926-e20fd94fd9dc', 'status': 'completed', 'code': 200} May 20 04:27:41.591012 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-dd1a7f03-d88a-4839-8166-56c522ca2f29 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:41.592875 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-dd1a7f03-d88a-4839-8166-56c522ca2f29 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=c57d56efd5fb40069159f50973655a5d&device_id=dfb7dc32-7bb4-4030-890b-55321eae3099 HTTP/1.1" status: 200 len: 1096 time: 0.0766604 May 20 04:27:41.662623 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-5acf89c0-3b03-4278-9f0c-b0d68af67b70 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=95c6e357-0f6b-489a-b926-e20fd94fd9dc HTTP/1.1" status: 200 len: 217 time: 0.0592244 May 20 04:27:41.726461 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-2dc8d6c9-6491-4376-9407-639ad041373e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:41.727897 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-2dc8d6c9-6491-4376-9407-639ad041373e service neutron] 10.222.0.74 "GET /v2.0/subnets?id=cb4fcf1d-e5d6-469d-83a3-5d90ce39e474 HTTP/1.1" status: 200 len: 848 time: 0.0594580 May 20 04:27:41.781207 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-b6f2b80d-bae0-4070-b442-613e3fc1a57d service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=e3248a7c-d6dc-404f-9bb7-8e87d070d1cb&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0406122 May 20 04:27:41.999881 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-5791d89b-06f3-4aaf-8b0c-31394e0fa227 service neutron] 10.222.0.74 "GET /v2.0/networks/e3248a7c-d6dc-404f-9bb7-8e87d070d1cb?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2085497 May 20 04:27:42.143488 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b1df8d43-16e1-4f5e-8847-4f95c0a79b25 req-22ae159b-59a8-401b-9741-da52eb1ac709 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:42.145000 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b1df8d43-16e1-4f5e-8847-4f95c0a79b25 req-22ae159b-59a8-401b-9741-da52eb1ac709 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.1027582 May 20 04:27:42.166573 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-10ccb85c-0a09-4658-8a64-d80325261c1f req-0e349356-ee1c-45aa-9c40-26091f7a098b service neutron] 10.222.0.74 "GET /v2.0/networks/e3248a7c-d6dc-404f-9bb7-8e87d070d1cb?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1505806 May 20 04:27:42.246546 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b1df8d43-16e1-4f5e-8847-4f95c0a79b25 req-8e378496-cf72-4a52-8944-4bbdc6b78072 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0634682 May 20 04:27:42.257194 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 40948) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:27:42.279310 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8c79d30f-4041-4dd3-8d1f-7c0d349bca12 req-741208c6-e7b9-4351-b9cc-629be9f98e96 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:42.280728 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8c79d30f-4041-4dd3-8d1f-7c0d349bca12 req-741208c6-e7b9-4351-b9cc-629be9f98e96 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfb7dc32-7bb4-4030-890b-55321eae3099 HTTP/1.1" status: 200 len: 943 time: 0.1348462 May 20 04:27:42.298716 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f8de4d2e-4a1b-4363-9a8c-54d639d11dd1 req-c98a44b6-ca59-4865-8223-82d391337c89 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:42.300226 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f8de4d2e-4a1b-4363-9a8c-54d639d11dd1 req-c98a44b6-ca59-4865-8223-82d391337c89 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=c57d56efd5fb40069159f50973655a5d&device_id=dfb7dc32-7bb4-4030-890b-55321eae3099 HTTP/1.1" status: 200 len: 1096 time: 0.0880516 May 20 04:27:42.345471 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8c79d30f-4041-4dd3-8d1f-7c0d349bca12 req-4102e1a5-dc11-4ce3-9db5-50f5d11953e2 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0562301 May 20 04:27:42.459930 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-9d3d0cf6-bb78-4a56-8ef2-75afd8c28c93 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=0484029d05714b4d860a9e429b5e7d70&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2010436 May 20 04:27:42.461863 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 40958) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:27:42.462393 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f8de4d2e-4a1b-4363-9a8c-54d639d11dd1 req-b928dce0-cc6e-490b-8289-faad3b08d88c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:42.464354 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f8de4d2e-4a1b-4363-9a8c-54d639d11dd1 req-b928dce0-cc6e-490b-8289-faad3b08d88c service neutron] 10.222.0.74 "GET /v2.0/networks?id=e3248a7c-d6dc-404f-9bb7-8e87d070d1cb HTTP/1.1" status: 200 len: 909 time: 0.1491520 May 20 04:27:42.524034 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6fd7ba7a-bf3b-45ac-ad71-602708bead2a admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:42.536840 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f8de4d2e-4a1b-4363-9a8c-54d639d11dd1 req-8253a102-22eb-40cc-a1da-330167b3139a service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=95c6e357-0f6b-489a-b926-e20fd94fd9dc HTTP/1.1" status: 200 len: 217 time: 0.0654204 May 20 04:27:42.589105 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6fd7ba7a-bf3b-45ac-ad71-602708bead2a admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:42.601803 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f8de4d2e-4a1b-4363-9a8c-54d639d11dd1 req-df94663f-663a-40b7-87f6-977cb847bedb service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:42.603326 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f8de4d2e-4a1b-4363-9a8c-54d639d11dd1 req-df94663f-663a-40b7-87f6-977cb847bedb service neutron] 10.222.0.74 "GET /v2.0/subnets?id=cb4fcf1d-e5d6-469d-83a3-5d90ce39e474 HTTP/1.1" status: 200 len: 848 time: 0.0607176 May 20 04:27:42.616891 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-df5210c4-2c76-4bd9-bf6c-1b3baea5f388 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:42.650449 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f8de4d2e-4a1b-4363-9a8c-54d639d11dd1 req-0c273d42-63d3-4adf-a059-f9ab3f8e2109 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=e3248a7c-d6dc-404f-9bb7-8e87d070d1cb&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0405080 May 20 04:27:42.718829 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b6c1d1e2-19b5-4c31-8f48-c1ab7a8072c7 req-954fdd84-c306-47fa-bc95-59b9331100d9 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:42.720306 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b6c1d1e2-19b5-4c31-8f48-c1ab7a8072c7 req-954fdd84-c306-47fa-bc95-59b9331100d9 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.1819334 May 20 04:27:42.726978 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d257195f-eb27-421f-900f-21f82882f8cc req-21bcf75d-a69c-4212-befc-e4de33586be6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:42.728425 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d257195f-eb27-421f-900f-21f82882f8cc req-21bcf75d-a69c-4212-befc-e4de33586be6 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=198bc8ab083f4a729f32b8ef27a6ee77&device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 1098 time: 0.1323807 May 20 04:27:42.741895 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6fd7ba7a-bf3b-45ac-ad71-602708bead2a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:42.748879 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-76990c91-557a-40cb-984d-21e679e1bb19 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:42.792625 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f8de4d2e-4a1b-4363-9a8c-54d639d11dd1 req-9669c2a5-a74e-423d-be1e-2c6b4de54a9d service neutron] 10.222.0.74 "GET /v2.0/networks/e3248a7c-d6dc-404f-9bb7-8e87d070d1cb?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1257071 May 20 04:27:42.800601 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6fd7ba7a-bf3b-45ac-ad71-602708bead2a admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=1b091fdd-0a0f-46fb-8de3-aea8fa6fdde8,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:42.810678 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-6fd7ba7a-bf3b-45ac-ad71-602708bead2a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.062s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:42.881262 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6fd7ba7a-bf3b-45ac-ad71-602708bead2a admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:42.882388 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-6fd7ba7a-bf3b-45ac-ad71-602708bead2a admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/1b091fdd-0a0f-46fb-8de3-aea8fa6fdde8 HTTP/1.1" status: 204 len: 168 time: 0.4192615 May 20 04:27:42.888011 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b6c1d1e2-19b5-4c31-8f48-c1ab7a8072c7 req-bc5687a0-a609-4cfe-9b68-c3739f9cdecd tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1560805 May 20 04:27:42.929106 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d257195f-eb27-421f-900f-21f82882f8cc req-54f72e23-2c6e-4b0d-a165-d9a0f77338b5 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:42.930613 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d257195f-eb27-421f-900f-21f82882f8cc req-54f72e23-2c6e-4b0d-a165-d9a0f77338b5 service neutron] 10.222.0.74 "GET /v2.0/networks?id=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d HTTP/1.1" status: 200 len: 910 time: 0.1851392 May 20 04:27:42.938367 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f8de4d2e-4a1b-4363-9a8c-54d639d11dd1 req-7d24385e-07a1-4e2f-8f70-6a954e5baa44 service neutron] 10.222.0.74 "GET /v2.0/networks/e3248a7c-d6dc-404f-9bb7-8e87d070d1cb?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1379907 May 20 04:27:43.035442 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d257195f-eb27-421f-900f-21f82882f8cc req-2a08168f-9de3-4420-bc52-06754de71257 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=2cf59c2c-723c-4450-8b98-0809b16290a8 HTTP/1.1" status: 200 len: 1001 time: 0.0978403 May 20 04:27:43.106830 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d257195f-eb27-421f-900f-21f82882f8cc req-b060f8c0-9967-4e0c-a415-ebb875378429 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:43.108353 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d257195f-eb27-421f-900f-21f82882f8cc req-b060f8c0-9967-4e0c-a415-ebb875378429 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3 HTTP/1.1" status: 200 len: 848 time: 0.0665541 May 20 04:27:43.189049 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d257195f-eb27-421f-900f-21f82882f8cc req-339d765c-c99f-46be-ab6e-3a9d2ff1b0ff service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:43.190525 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d257195f-eb27-421f-900f-21f82882f8cc req-339d765c-c99f-46be-ab6e-3a9d2ff1b0ff service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0744607 May 20 04:27:43.322626 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d257195f-eb27-421f-900f-21f82882f8cc req-a3b44f28-928b-4716-8c80-e7bb711bf8bf service neutron] 10.222.0.74 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1245601 May 20 04:27:43.491263 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d257195f-eb27-421f-900f-21f82882f8cc req-f8dae4cb-2599-4664-ae7c-f5abdb6c4dc4 service neutron] 10.222.0.74 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1614871 May 20 04:27:43.540001 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0cb36a19-c0e7-466a-bc53-19476ca0ea9a req-3ad9e3ea-876d-4a4a-b8d7-184038c4c896 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:43.541450 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0cb36a19-c0e7-466a-bc53-19476ca0ea9a req-3ad9e3ea-876d-4a4a-b8d7-184038c4c896 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfb7dc32-7bb4-4030-890b-55321eae3099 HTTP/1.1" status: 200 len: 943 time: 0.0900736 May 20 04:27:43.669503 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0cb36a19-c0e7-466a-bc53-19476ca0ea9a req-d1e8976c-93f5-41d0-84c0-f4f25d8dfa72 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1222043 May 20 04:27:44.080613 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e3901b66-4251-45bd-91c9-32b058c46f3e req-40f4d3e1-3ce6-4abf-a5ba-b7b921c9a9a9 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:44.082434 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e3901b66-4251-45bd-91c9-32b058c46f3e req-40f4d3e1-3ce6-4abf-a5ba-b7b921c9a9a9 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0836341 May 20 04:27:44.131506 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e3901b66-4251-45bd-91c9-32b058c46f3e req-d6fd57ff-d5f8-45f9-bc28-f6048bfe6d30 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0427675 May 20 04:27:44.872932 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f2db5f23-5fd3-429a-81b0-f93dc7188061 req-8b191ae6-d8a9-47a5-90fb-996cfb615063 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:44.874365 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f2db5f23-5fd3-429a-81b0-f93dc7188061 req-8b191ae6-d8a9-47a5-90fb-996cfb615063 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfb7dc32-7bb4-4030-890b-55321eae3099 HTTP/1.1" status: 200 len: 943 time: 0.0912473 May 20 04:27:44.920789 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f2db5f23-5fd3-429a-81b0-f93dc7188061 req-8d1db7e4-ab64-4027-a622-dd32330e3b61 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0379014 May 20 04:27:44.981080 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:45.096957 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-56e67302-9d7f-4adc-8402-a41970278914 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:45.329574 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f33e4d32-bea5-4aff-889a-d7f82543c4cf req-f5072ea9-394f-4d1a-87ad-8e86747d21bc tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:45.330976 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f33e4d32-bea5-4aff-889a-d7f82543c4cf req-f5072ea9-394f-4d1a-87ad-8e86747d21bc tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0882912 May 20 04:27:45.375247 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f33e4d32-bea5-4aff-889a-d7f82543c4cf req-9395b51f-afe3-4462-9648-c0d7418b1995 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0384104 May 20 04:27:46.119190 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9e2f845e-8b9f-4d95-bde9-d0a61bb30fea req-94cbff61-f268-43ed-9c6b-d3f5a14291d1 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:46.120768 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9e2f845e-8b9f-4d95-bde9-d0a61bb30fea req-94cbff61-f268-43ed-9c6b-d3f5a14291d1 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfb7dc32-7bb4-4030-890b-55321eae3099 HTTP/1.1" status: 200 len: 943 time: 0.0868354 May 20 04:27:46.165556 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9e2f845e-8b9f-4d95-bde9-d0a61bb30fea req-9ec24980-e6fe-41b4-bff6-908a52e78e18 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0387089 May 20 04:27:46.187323 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:47.355803 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d71a6d42-7a6c-4e58-bbb5-fa16889f0df7 req-d159bc13-3431-4344-bfbe-8bb32869bb99 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:47.357068 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d71a6d42-7a6c-4e58-bbb5-fa16889f0df7 req-d159bc13-3431-4344-bfbe-8bb32869bb99 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfb7dc32-7bb4-4030-890b-55321eae3099 HTTP/1.1" status: 200 len: 943 time: 0.0835721 May 20 04:27:47.378788 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 95c6e357-0f6b-489a-b926-e20fd94fd9dc details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:27:47.403160 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d71a6d42-7a6c-4e58-bbb5-fa16889f0df7 req-a68eab10-0b4d-4854-ae21-5ed9912da893 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0402331 May 20 04:27:47.525846 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 95c6e357-0f6b-489a-b926-e20fd94fd9dc, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=95c6e357-0f6b-489a-b926-e20fd94fd9dc,segment=NetworkSegment(0d9a1ad5-c1fc-43c7-8293-6032c9456478),segment_id=0d9a1ad5-c1fc-43c7-8293-6032c9456478)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:27:47.546656 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': '95c6e357-0f6b-489a-b926-e20fd94fd9dc', 'network_id': 'e3248a7c-d6dc-404f-9bb7-8e87d070d1cb', 'port_id': '95c6e357-0f6b-489a-b926-e20fd94fd9dc', 'mac_address': 'fa:16:3e:61:d4:9c', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 809, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'cb4fcf1d-e5d6-469d-83a3-5d90ce39e474', 'ip_address': '172.20.0.3'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:27:47.602316 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:47.619667 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 95c6e357-0f6b-489a-b926-e20fd94fd9dc {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:27:47.636306 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 95c6e357-0f6b-489a-b926-e20fd94fd9dc, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=95c6e357-0f6b-489a-b926-e20fd94fd9dc,segment=NetworkSegment(0d9a1ad5-c1fc-43c7-8293-6032c9456478),segment_id=0d9a1ad5-c1fc-43c7-8293-6032c9456478)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:27:47.643845 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:47.645621 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:47.711848 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=95c6e357-0f6b-489a-b926-e20fd94fd9dc,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:47.714757 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:47.836218 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:48.059479 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['95c6e357-0f6b-489a-b926-e20fd94fd9dc'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:27:48.615672 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-20b0ed96-647d-41b3-8067-93caf3f69643 req-ec84cfea-6dba-432e-86c3-27217bfb3051 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:48.617108 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-20b0ed96-647d-41b3-8067-93caf3f69643 req-ec84cfea-6dba-432e-86c3-27217bfb3051 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfb7dc32-7bb4-4030-890b-55321eae3099 HTTP/1.1" status: 200 len: 944 time: 0.0846722 May 20 04:27:48.667298 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-20b0ed96-647d-41b3-8067-93caf3f69643 req-54341443-7884-45e7-beb0-fe35d14ae170 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0417309 May 20 04:27:49.412115 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 95c6e357-0f6b-489a-b926-e20fd94fd9dc up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:27:49.491996 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port 95c6e357-0f6b-489a-b926-e20fd94fd9dc completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:27:49.496682 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port 95c6e357-0f6b-489a-b926-e20fd94fd9dc triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:27:49.497081 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:49.624232 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:49.651963 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'dfb7dc32-7bb4-4030-890b-55321eae3099', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '95c6e357-0f6b-489a-b926-e20fd94fd9dc'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:27:49.662471 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 95c6e357-0f6b-489a-b926-e20fd94fd9dc, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=95c6e357-0f6b-489a-b926-e20fd94fd9dc,segment=NetworkSegment(0d9a1ad5-c1fc-43c7-8293-6032c9456478),segment_id=0d9a1ad5-c1fc-43c7-8293-6032c9456478)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:27:49.668573 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:49.670462 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-8d7c0303-a603-4ea9-a079-08b39409da86 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:49.737431 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=95c6e357-0f6b-489a-b926-e20fd94fd9dc,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:49.740462 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.070s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:49.866077 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ff5c4b6c-9e4c-4cdb-bd13-ed3b6bac2b87 req-578bfd77-bc76-4c67-a2f9-6c5816402e90 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:49.867527 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ff5c4b6c-9e4c-4cdb-bd13-ed3b6bac2b87 req-578bfd77-bc76-4c67-a2f9-6c5816402e90 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfb7dc32-7bb4-4030-890b-55321eae3099 HTTP/1.1" status: 200 len: 945 time: 0.0869520 May 20 04:27:49.893724 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'dfb7dc32-7bb4-4030-890b-55321eae3099', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '95c6e357-0f6b-489a-b926-e20fd94fd9dc', 'code': 200} May 20 04:27:49.914504 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ff5c4b6c-9e4c-4cdb-bd13-ed3b6bac2b87 req-5ac7e5c6-c98d-4fb2-bf5c-7eeab22d088c tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0408456 May 20 04:27:50.155072 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-91c82374-22a3-4265-bfce-a65c438ca6b4 req-f6d8604f-0afd-488b-95ab-db8a42b943b6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:50.156597 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-91c82374-22a3-4265-bfce-a65c438ca6b4 req-f6d8604f-0afd-488b-95ab-db8a42b943b6 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=dfb7dc32-7bb4-4030-890b-55321eae3099 HTTP/1.1" status: 200 len: 1098 time: 0.0750957 May 20 04:27:50.198067 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-91c82374-22a3-4265-bfce-a65c438ca6b4 req-5bcf18be-c931-42fc-b4bf-8c2dda894fb9 service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0350778 May 20 04:27:51.130671 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ed6b7675-2b32-45a7-a2e2-45e5f1f8c470 req-25c44925-7ab5-4ac8-a73d-812a1adcf42d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:51.132153 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ed6b7675-2b32-45a7-a2e2-45e5f1f8c470 req-25c44925-7ab5-4ac8-a73d-812a1adcf42d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfb7dc32-7bb4-4030-890b-55321eae3099 HTTP/1.1" status: 200 len: 945 time: 0.0994220 May 20 04:27:51.176655 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ed6b7675-2b32-45a7-a2e2-45e5f1f8c470 req-f9ae768f-b495-4a3e-a9a8-ae8404571450 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0386214 May 20 04:27:52.391121 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7bfc3c93-92cc-49de-af47-daf1bc98bd3e req-a9a5e6a0-cbcc-4f98-9960-b2d4030939d1 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:52.392156 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7bfc3c93-92cc-49de-af47-daf1bc98bd3e req-a9a5e6a0-cbcc-4f98-9960-b2d4030939d1 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfb7dc32-7bb4-4030-890b-55321eae3099 HTTP/1.1" status: 200 len: 945 time: 0.0978236 May 20 04:27:52.437228 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7bfc3c93-92cc-49de-af47-daf1bc98bd3e req-731f10a5-389f-4ff5-bbbc-ecc939cc4ac5 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0393498 May 20 04:27:53.620512 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-736d8256-7c24-4d18-8b27-671ce2689198 req-3c45e876-edbb-4dec-98c1-303bf6c23583 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:53.621925 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-736d8256-7c24-4d18-8b27-671ce2689198 req-3c45e876-edbb-4dec-98c1-303bf6c23583 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfb7dc32-7bb4-4030-890b-55321eae3099 HTTP/1.1" status: 200 len: 945 time: 0.0845616 May 20 04:27:53.669073 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-736d8256-7c24-4d18-8b27-671ce2689198 req-16af3af9-32aa-47c1-b6d6-25b338434398 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0413353 May 20 04:27:54.869530 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a184eb41-071c-4870-b696-7807b12de879 req-28b2906e-fcba-4111-9d54-23431750d86f tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:54.870970 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a184eb41-071c-4870-b696-7807b12de879 req-28b2906e-fcba-4111-9d54-23431750d86f tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfb7dc32-7bb4-4030-890b-55321eae3099 HTTP/1.1" status: 200 len: 945 time: 0.0844655 May 20 04:27:54.917344 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a184eb41-071c-4870-b696-7807b12de879 req-a0462bfc-c46e-48ed-a2c1-c1b929f532f2 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/security-groups?id=e551ae97-21ce-4dbb-b450-b8fe6812efec&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0393472 May 20 04:27:55.372705 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4237edc3-bfee-4ba4-a7fd-eba00a7702c6 req-8cb87c3d-ffd7-4d85-bd16-4273d352b2a1 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:27:55.373994 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4237edc3-bfee-4ba4-a7fd-eba00a7702c6 req-8cb87c3d-ffd7-4d85-bd16-4273d352b2a1 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.74 "GET /v2.0/ports?device_id=dfb7dc32-7bb4-4030-890b-55321eae3099 HTTP/1.1" status: 200 len: 945 time: 0.0856874 May 20 04:27:55.510343 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-4237edc3-bfee-4ba4-a7fd-eba00a7702c6 req-3959a328-08c8-4edb-b86d-9d78a2929cd2 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Deleting port 95c6e357-0f6b-489a-b926-e20fd94fd9dc {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:27:55.510636 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4237edc3-bfee-4ba4-a7fd-eba00a7702c6 req-3959a328-08c8-4edb-b86d-9d78a2929cd2 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:55.658439 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-4237edc3-bfee-4ba4-a7fd-eba00a7702c6 req-3959a328-08c8-4edb-b86d-9d78a2929cd2 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] For port 95c6e357-0f6b-489a-b926-e20fd94fd9dc, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=95c6e357-0f6b-489a-b926-e20fd94fd9dc,segment=NetworkSegment(0d9a1ad5-c1fc-43c7-8293-6032c9456478),segment_id=0d9a1ad5-c1fc-43c7-8293-6032c9456478)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:27:55.658700 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4237edc3-bfee-4ba4-a7fd-eba00a7702c6 req-3959a328-08c8-4edb-b86d-9d78a2929cd2 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:55.869530 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-4237edc3-bfee-4ba4-a7fd-eba00a7702c6 req-3959a328-08c8-4edb-b86d-9d78a2929cd2 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Calling delete_port for 95c6e357-0f6b-489a-b926-e20fd94fd9dc owned by compute:nova {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:27:55.979286 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4237edc3-bfee-4ba4-a7fd-eba00a7702c6 req-3959a328-08c8-4edb-b86d-9d78a2929cd2 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:55.985125 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-ac065659-71c5-49c0-ac75-d87375539664 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:27:55.997834 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['95c6e357-0f6b-489a-b926-e20fd94fd9dc'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:27:56.036301 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3959a328-08c8-4edb-b86d-9d78a2929cd2 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Pushing event deleted for resources: {'Port': ['ID=95c6e357-0f6b-489a-b926-e20fd94fd9dc,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:27:56.068580 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-3959a328-08c8-4edb-b86d-9d78a2929cd2 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.083s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:27:56.108361 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fb0e42e0-43b5-4548-9154-17b2569cd6a6 req-da4669f0-4e49-41d5-80c7-fa6ea59a1542 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfb7dc32-7bb4-4030-890b-55321eae3099 HTTP/1.1" status: 200 len: 210 time: 0.0812156 May 20 04:27:56.151055 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [req-4237edc3-bfee-4ba4-a7fd-eba00a7702c6 req-3959a328-08c8-4edb-b86d-9d78a2929cd2 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] No DVR routers for this DVR port 95c6e357-0f6b-489a-b926-e20fd94fd9dc on host hv-cinder-78240 {{(pid=91643) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:27:56.153690 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4237edc3-bfee-4ba4-a7fd-eba00a7702c6 req-3959a328-08c8-4edb-b86d-9d78a2929cd2 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:27:56.156382 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'dfb7dc32-7bb4-4030-890b-55321eae3099', 'name': 'network-vif-deleted', 'tag': '95c6e357-0f6b-489a-b926-e20fd94fd9dc'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:27:56.168040 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4237edc3-bfee-4ba4-a7fd-eba00a7702c6 req-3959a328-08c8-4edb-b86d-9d78a2929cd2 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] 10.222.0.74 "DELETE /v2.0/ports/95c6e357-0f6b-489a-b926-e20fd94fd9dc HTTP/1.1" status: 204 len: 173 time: 0.7872565 May 20 04:27:56.201323 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'dfb7dc32-7bb4-4030-890b-55321eae3099', 'name': 'network-vif-deleted', 'tag': '95c6e357-0f6b-489a-b926-e20fd94fd9dc', 'status': 'completed', 'code': 200} May 20 04:28:00.621322 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 95c6e357-0f6b-489a-b926-e20fd94fd9dc no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:28:00.656042 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: 95c6e357-0f6b-489a-b926-e20fd94fd9dc {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:28:00.656386 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 95c6e357-0f6b-489a-b926-e20fd94fd9dc not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:28:05.598926 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:07.749431 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-98812566-aeaa-460c-97ef-1e804dc9b6c2 req-dffb8b84-8347-4f60-9685-68288930e395 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0717709 May 20 04:28:07.906302 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-98812566-aeaa-460c-97ef-1e804dc9b6c2 req-d5ef78d4-1e82-4cb4-ad7a-5371f3c9db78 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:07.908192 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-98812566-aeaa-460c-97ef-1e804dc9b6c2 req-d5ef78d4-1e82-4cb4-ad7a-5371f3c9db78 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=9fafdf3a29024ac588e5b24719735677&device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 1099 time: 0.0805125 May 20 04:28:08.062787 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-98812566-aeaa-460c-97ef-1e804dc9b6c2 req-a652fa16-4690-416c-839d-384c23fc1059 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:08.064664 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-98812566-aeaa-460c-97ef-1e804dc9b6c2 req-a652fa16-4690-416c-839d-384c23fc1059 service neutron] 10.222.0.74 "GET /v2.0/networks?id=64bcea8a-6d35-406b-a823-fa1185008174 HTTP/1.1" status: 200 len: 918 time: 0.1349061 May 20 04:28:08.207107 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-98812566-aeaa-460c-97ef-1e804dc9b6c2 req-8b864920-bbfe-4036-9ba1-57f3096b8d7d service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=535d5637-ae66-43cb-bd16-3543bd3b7a8b HTTP/1.1" status: 200 len: 1001 time: 0.1354849 May 20 04:28:08.279870 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-98812566-aeaa-460c-97ef-1e804dc9b6c2 req-18bb8002-a2d6-4c2d-993c-82dd00363a27 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:08.281503 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-98812566-aeaa-460c-97ef-1e804dc9b6c2 req-18bb8002-a2d6-4c2d-993c-82dd00363a27 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=203a3962-2282-4f51-ac92-c9695b5c0cfe HTTP/1.1" status: 200 len: 856 time: 0.0665843 May 20 04:28:08.362947 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-98812566-aeaa-460c-97ef-1e804dc9b6c2 req-06804755-33a1-43e1-9cda-a44950b34ce2 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:08.364392 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-98812566-aeaa-460c-97ef-1e804dc9b6c2 req-06804755-33a1-43e1-9cda-a44950b34ce2 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=64bcea8a-6d35-406b-a823-fa1185008174&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0737567 May 20 04:28:08.510423 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-98812566-aeaa-460c-97ef-1e804dc9b6c2 req-0ec551d0-0903-4614-9b66-c73720a7c487 service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1383224 May 20 04:28:08.692482 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-98812566-aeaa-460c-97ef-1e804dc9b6c2 req-b1b3b515-bcff-4a73-8f23-ac59dbaa7f63 service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1723540 May 20 04:28:12.621453 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-160e9130-8b12-4448-bc8e-a5445cc6a8a7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:12.999684 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:28:14.981493 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:15.103333 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-11c63293-58e5-43a2-a236-f4389c4008fc None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:16.190541 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:17.831647 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:21.314603 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 41252) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:28:21.383349 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_v2 [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] Deleting subnet cb4fcf1d-e5d6-469d-83a3-5d90ce39e474 {{(pid=91643) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 20 04:28:21.386321 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:21.439516 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-1472754'] for subnet, precommit_delete_associations {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:21.502494 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-7786687'] for subnet, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:21.747967 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-7786632', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2050499', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2172490'] for subnet, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:21.752541 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:21.785678 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=cb4fcf1d-e5d6-469d-83a3-5d90ce39e474,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:21.788065 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:21.796929 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:21.797297 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:21.797656 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:21.798002 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:21.798347 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:21.798695 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:21.799035 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:21.799404 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:21.799741 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:21.800088 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:21.800433 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:21.800764 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:21.804305 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] 10.222.0.39 "DELETE /v2.0/subnets/cb4fcf1d-e5d6-469d-83a3-5d90ce39e474 HTTP/1.1" status: 204 len: 168 time: 0.4886935 May 20 04:28:21.805151 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-f11ef1c0-756b-4cf8-9795-48f57c738bb7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:21.806784 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 41256) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:28:21.998387 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-985a87e6-5d0c-4aa8-947d-407ef0f66109 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-593607'] for network, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:22.122398 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-985a87e6-5d0c-4aa8-947d-407ef0f66109 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-7789108', 'neutron.services.segments.db._delete_segments_for_network-8760688958082'] for network, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:22.138993 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-985a87e6-5d0c-4aa8-947d-407ef0f66109 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['e3248a7c-d6dc-404f-9bb7-8e87d070d1cb']}} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:22.151047 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-985a87e6-5d0c-4aa8-947d-407ef0f66109 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '0d9a1ad5-c1fc-43c7-8293-6032c9456478') {'for_net_delete': True} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:22.151379 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-985a87e6-5d0c-4aa8-947d-407ef0f66109 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '0d9a1ad5-c1fc-43c7-8293-6032c9456478') {} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:22.162471 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-985a87e6-5d0c-4aa8-947d-407ef0f66109 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8760688968331', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-1472741'] for segment, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:22.179553 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-985a87e6-5d0c-4aa8-947d-407ef0f66109 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:22.189737 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-985a87e6-5d0c-4aa8-947d-407ef0f66109 admin admin] Releasing vlan 809 on physical network physnet1 to pool {{(pid=91643) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 20 04:28:22.190317 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-985a87e6-5d0c-4aa8-947d-407ef0f66109 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2070625'] for segment, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: Traceback (most recent call last): May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: timer() May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: cb(*args, **kw) May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: return func(*args, **kwargs) May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: self._notify() May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: self.callback(batched_events) May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: event.method(event) May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: aggregate_id = self._get_aggregate_id(event.segment_id) May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: aggregate_uuid = self.p_client.list_aggregates( May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: return f(self, *a, **k) May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: return self._get(url).json() May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: return self._client.get(url, endpoint_filter=self._ks_filter, May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: return self.request(url, 'GET', **kwargs) May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: base_url = self.get_endpoint(auth, allow=allow, May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: auth = self._auth_required(auth, 'determine endpoint URL') May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 20 04:28:22.207738 dv-cinder-782405-9-iscsi neutron-server[91643]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 20 04:28:22.285296 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-985a87e6-5d0c-4aa8-947d-407ef0f66109 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-7789117', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-8053321'] for network, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:22.292322 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-82159594-298b-415e-b998-0ccc1080748a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:22.310759 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-985a87e6-5d0c-4aa8-947d-407ef0f66109 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:22.320743 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-985a87e6-5d0c-4aa8-947d-407ef0f66109 admin admin] Pushing event deleted for resources: {'Network': ['ID=e3248a7c-d6dc-404f-9bb7-8e87d070d1cb,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:22.323754 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-985a87e6-5d0c-4aa8-947d-407ef0f66109 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.032s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:22.330613 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-985a87e6-5d0c-4aa8-947d-407ef0f66109 admin admin] 10.222.0.39 "DELETE /v2.0/networks/e3248a7c-d6dc-404f-9bb7-8e87d070d1cb HTTP/1.1" status: 204 len: 168 time: 0.5229065 May 20 04:28:22.526094 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 41264) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:28:22.577599 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-27ff27a0-f29c-49da-9943-d487daea755f admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=c57d56efd5fb40069159f50973655a5d&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0495124 May 20 04:28:22.580260 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 41268) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:28:22.639496 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8d49c95b-ba15-43b5-bb02-6cc4515a16fe admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:22.665446 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8d49c95b-ba15-43b5-bb02-6cc4515a16fe admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:22.726323 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8d49c95b-ba15-43b5-bb02-6cc4515a16fe admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:22.730045 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-6fd7ba7a-bf3b-45ac-ad71-602708bead2a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:22.743284 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8d49c95b-ba15-43b5-bb02-6cc4515a16fe admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e551ae97-21ce-4dbb-b450-b8fe6812efec,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:22.745848 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-8d49c95b-ba15-43b5-bb02-6cc4515a16fe admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:22.754557 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8d49c95b-ba15-43b5-bb02-6cc4515a16fe admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:22.755679 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-8d49c95b-ba15-43b5-bb02-6cc4515a16fe admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/e551ae97-21ce-4dbb-b450-b8fe6812efec HTTP/1.1" status: 204 len: 168 time: 0.1745684 May 20 04:28:23.525672 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 41292) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:28:23.651694 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Request body: {'network': {'name': 'tempest-TestMinimumBasicScenario-22498061-network', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:28:23.656454 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:28:23.660699 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Usage tracker for resource:network and tenant:13d0e12c764f483ea457246b10c4cf7e is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:28:23.663755 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:28:23.670352 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:28:23.680853 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Made reservation on behalf of 13d0e12c764f483ea457246b10c4cf7e for: {'network': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:28:23.683088 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-585852'] for network, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:23.698280 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:23.712741 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:28:23.717623 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Usage tracker for resource:security_group_rule and tenant:13d0e12c764f483ea457246b10c4cf7e is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:28:23.721726 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:28:23.730231 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:28:23.819570 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:23.825291 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:23.828372 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-d37dd146-40c8-4530-82e9-7f4b30095805 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:23.856209 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 801} {{(pid=91643) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 20 04:28:23.865376 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f596c737-91de-4c7f-9aa1-e6e95d668a70,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:23.868552 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.040s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:23.872786 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:23.873176 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.segments_db [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Added segment d558b057-1c72-4efe-ab53-567b572e2de5 of type vlan for network 9798d104-d4de-413f-b41d-fb9d7e8a818e May 20 04:28:23.899628 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8760688516720'] for network, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:23.930508 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915'] for network, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:23.931831 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:23.932408 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:23.936994 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:24.028900 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] 10.222.0.39 "POST /v2.0/networks HTTP/1.1" status: 201 len: 886 time: 0.5021608 May 20 04:28:24.031935 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 41298) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:28:24.039435 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-14d0e401-9769-46a4-8209-27b26f31a2c9 admin admin] Request body: {'subnet': {'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestMinimumBasicScenario-22498061-subnet', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'ip_version': 4}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:28:24.042461 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Pushing event updated for resources: {'Network': ['ID=9798d104-d4de-413f-b41d-fb9d7e8a818e,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:24.045702 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.109s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:24.046620 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-14d0e401-9769-46a4-8209-27b26f31a2c9 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:28:24.051278 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-14d0e401-9769-46a4-8209-27b26f31a2c9 admin admin] Usage tracker for resource:subnet and tenant:13d0e12c764f483ea457246b10c4cf7e is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:28:24.053000 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Network 9798d104-d4de-413f-b41d-fb9d7e8a818e details requested by agent ovs-agent-dv-cinder-782405-9-iscsi with host dv-cinder-782405-9-iscsi {{(pid=91645) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 20 04:28:24.053966 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-14d0e401-9769-46a4-8209-27b26f31a2c9 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:28:24.059886 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-14d0e401-9769-46a4-8209-27b26f31a2c9 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:28:24.069554 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-14d0e401-9769-46a4-8209-27b26f31a2c9 admin admin] Made reservation on behalf of 13d0e12c764f483ea457246b10c4cf7e for: {'subnet': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:28:24.072058 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-14d0e401-9769-46a4-8209-27b26f31a2c9 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2172529'] for subnet, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:24.468583 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-14d0e401-9769-46a4-8209-27b26f31a2c9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-2050444', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2172483'] for subnet, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:24.474098 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:24.540691 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-14d0e401-9769-46a4-8209-27b26f31a2c9 admin admin] Pushing event updated for resources: {'Subnet': ['ID=c5423164-3d24-4562-871f-ab03914420dc,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:24.545456 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-14d0e401-9769-46a4-8209-27b26f31a2c9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:24.554760 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-14d0e401-9769-46a4-8209-27b26f31a2c9 admin admin] Checking segment: {'id': 'd558b057-1c72-4efe-ab53-567b572e2de5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 801, 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91643) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:28:24.555112 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-14d0e401-9769-46a4-8209-27b26f31a2c9 admin admin] Network hostable DHCP agents. Network: 9798d104-d4de-413f-b41d-fb9d7e8a818e, hostable agents: ['fffae25a-f450-4182-8f9e-ac29d143f6c9'], hosted agents: [] {{(pid=91643) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 20 04:28:24.574903 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-14d0e401-9769-46a4-8209-27b26f31a2c9 admin admin] Network 9798d104-d4de-413f-b41d-fb9d7e8a818e is scheduled to be hosted by DHCP agent fffae25a-f450-4182-8f9e-ac29d143f6c9 with binding_index 1 {{(pid=91643) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 20 04:28:24.584640 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-14d0e401-9769-46a4-8209-27b26f31a2c9 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-782405-9-iscsi, 9798d104-d4de-413f-b41d-fb9d7e8a818e) {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:28:24.593033 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-14d0e401-9769-46a4-8209-27b26f31a2c9 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:24.593620 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-14d0e401-9769-46a4-8209-27b26f31a2c9 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:24.593895 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-dc739500-f349-4a7b-b60f-07b6d3ceed4e None None] Network 9798d104-d4de-413f-b41d-fb9d7e8a818e requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:28:24.615054 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-14d0e401-9769-46a4-8209-27b26f31a2c9 admin admin] 10.222.0.39 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 849 time: 0.5821273 May 20 04:28:24.617771 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 41302) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:28:24.622681 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Request body: {'router': {'name': 'tempest-TestMinimumBasicScenario-22498061-router', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:28:24.628110 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:28:24.632173 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Usage tracker for resource:router and tenant:13d0e12c764f483ea457246b10c4cf7e is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:28:24.635280 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:28:24.641406 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:28:24.653614 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-8071540', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-260712'] for router, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:24.654730 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-8056436', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-8053301', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-7881139', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-260707'] for router, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:24.670318 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:24.830535 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcp85c0b5c8-c799-5455-967d-3e9c2c8b4839-9798d104-d4de-413f-b41d-fb9d7e8a818e', 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'fixed_ips': [{'subnet_id': 'c5423164-3d24-4562-871f-ab03914420dc'}]}} from dv-cinder-782405-9-iscsi. {{(pid=91645) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 20 04:28:24.831379 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:24.987584 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.db_base_plugin_common [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Allocated IP 172.20.0.2 (9798d104-d4de-413f-b41d-fb9d7e8a818e/c5423164-3d24-4562-871f-ab03914420dc/617c2db0-1597-44e9-9e82-8ea22dfd47a8) {{(pid=91645) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:28:25.036592 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-8053308', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-8071430', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-260685'] for router, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:25.183936 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Notify callbacks [] for port, precommit_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:25.194031 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Transition to ACTIVE for port object 617c2db0-1597-44e9-9e82-8ea22dfd47a8 will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:28:25.212927 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Transition to ACTIVE for port object 617c2db0-1597-44e9-9e82-8ea22dfd47a8 will not be triggered until provisioned by entity DHCP. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:28:25.218872 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:25.223454 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:25.287990 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Network 9798d104-d4de-413f-b41d-fb9d7e8a818e is already hosted by enough agents. {{(pid=91645) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:28:25.290765 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.base_scheduler [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Resources bound (agent type, host, resource id): {{(pid=91645) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:28:25.293341 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Received port after_create {{(pid=91645) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:28:25.302324 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Pushing event updated for resources: {'Port': ['ID=617c2db0-1597-44e9-9e82-8ea22dfd47a8,revision_number=1']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:25.340739 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:25.340961 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:25.341287 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:25.341647 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:25.342285 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:25.342762 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:25.343086 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:25.343529 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:25.344503 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:25.344503 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:25.344503 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:25.344795 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:25.349697 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:25.372786 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.149s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:25.379429 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:25.402171 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Attempting to bind port 617c2db0-1597-44e9-9e82-8ea22dfd47a8 on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:28:25.402650 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Attempting to bind port 617c2db0-1597-44e9-9e82-8ea22dfd47a8 by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': 'd558b057-1c72-4efe-ab53-567b572e2de5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 801, 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:28:25.403012 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Attempting to bind port 617c2db0-1597-44e9-9e82-8ea22dfd47a8 on network 9798d104-d4de-413f-b41d-fb9d7e8a818e {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:28:25.408826 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 28, 15), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:28:25.409188 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Checking segment: {'id': 'd558b057-1c72-4efe-ab53-567b572e2de5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 801, 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:28:25.409582 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:25.409962 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Bound using segment: {'id': 'd558b057-1c72-4efe-ab53-567b572e2de5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 801, 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:28:25.410418 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Bound port: 617c2db0-1597-44e9-9e82-8ea22dfd47a8, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'd558b057-1c72-4efe-ab53-567b572e2de5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 801, 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:28:25.410784 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:25.497321 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] For port 617c2db0-1597-44e9-9e82-8ea22dfd47a8, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:28:25.500529 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] For port 617c2db0-1597-44e9-9e82-8ea22dfd47a8, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=617c2db0-1597-44e9-9e82-8ea22dfd47a8,segment=NetworkSegment(d558b057-1c72-4efe-ab53-567b572e2de5),segment_id=d558b057-1c72-4efe-ab53-567b572e2de5)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:28:25.505427 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:25.509871 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-dfc6f6f8-a39a-4ab9-a478-4083e76fe6c7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:25.518738 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Allocated IP 172.27.0.158 (712ce8c4-8feb-4719-bf13-e06992125cd9/e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5/160f1898-b408-446e-8492-c4b331489205) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:28:25.585343 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Pushing event updated for resources: {'Port': ['ID=617c2db0-1597-44e9-9e82-8ea22dfd47a8,revision_number=2']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:25.588450 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.079s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:25.672546 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:25.703174 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:25.706746 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-29d24236-7a5c-467d-9980-faeb65ffb525 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:25.769182 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:28:25.771984 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:28:25.775767 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:28:25.809218 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Pushing event updated for resources: {'Port': ['ID=160f1898-b408-446e-8492-c4b331489205,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:25.815522 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.109s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:25.981963 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 617c2db0-1597-44e9-9e82-8ea22dfd47a8 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:28:26.051204 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port 617c2db0-1597-44e9-9e82-8ea22dfd47a8 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:28:26.216643 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:26.533206 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-2d704309-05a4-4857-bed8-1dcae92a9bdd None None] Network 9798d104-d4de-413f-b41d-fb9d7e8a818e requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:28:26.636575 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-396589ca-e8a5-4b2d-8109-78f4c5e544d1 None None] Provisioning for port 617c2db0-1597-44e9-9e82-8ea22dfd47a8 completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:28:26.652375 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-396589ca-e8a5-4b2d-8109-78f4c5e544d1 None None] Provisioning complete for port 617c2db0-1597-44e9-9e82-8ea22dfd47a8 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:28:26.652781 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-396589ca-e8a5-4b2d-8109-78f4c5e544d1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:26.670156 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-8071551', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-8053391'] for router, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:26.860859 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-396589ca-e8a5-4b2d-8109-78f4c5e544d1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:26.876616 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Router 7b630b1e-4360-4f2e-9053-be7e631bfa3a is scheduled to L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 with binding_index 1 {{(pid=91643) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 20 04:28:26.900139 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-396589ca-e8a5-4b2d-8109-78f4c5e544d1 None None] For port 617c2db0-1597-44e9-9e82-8ea22dfd47a8, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=617c2db0-1597-44e9-9e82-8ea22dfd47a8,segment=NetworkSegment(d558b057-1c72-4efe-ab53-567b572e2de5),segment_id=d558b057-1c72-4efe-ab53-567b572e2de5)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:28:26.904629 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-396589ca-e8a5-4b2d-8109-78f4c5e544d1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:26.909106 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-a14e1d56-4a2f-40ff-aa8a-2790ce9d2ba7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:26.975770 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-396589ca-e8a5-4b2d-8109-78f4c5e544d1 None None] Pushing event updated for resources: {'Port': ['ID=617c2db0-1597-44e9-9e82-8ea22dfd47a8,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:26.978825 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-396589ca-e8a5-4b2d-8109-78f4c5e544d1 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.070s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:27.063602 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:28:27.072754 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Sync routers for ids ['7b630b1e-4360-4f2e-9053-be7e631bfa3a'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:28:27.083864 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Routers {'7b630b1e-4360-4f2e-9053-be7e631bfa3a'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:28:27.084475 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:28:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['7b630b1e-4360-4f2e-9053-be7e631bfa3a'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:27.089147 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Notify callbacks [] for router, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:27.093334 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] 10.222.0.39 "POST /v2.0/routers HTTP/1.1" status: 201 len: 925 time: 2.4717605 May 20 04:28:27.093532 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 41306) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:28:27.179999 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-0968ab9d-58d9-4e64-b9a7-53569d96ee18 None None] Provisioning complete for port 617c2db0-1597-44e9-9e82-8ea22dfd47a8 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:28:27.180262 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0968ab9d-58d9-4e64-b9a7-53569d96ee18 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:27.338809 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Set router 7b630b1e-4360-4f2e-9053-be7e631bfa3a gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:28:27.705860 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'name': 'tempest-TestMinimumBasicScenario-22498061-router', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158'}], 'enable_snat': True}, 'gw_port_id': '160f1898-b408-446e-8492-c4b331489205', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:24Z', 'updated_at': '2021-05-20T04:28:26Z', 'revision_number': 3, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'gw_port': {'id': '160f1898-b408-446e-8492-c4b331489205', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:79:e7:e5', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 544, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:25Z', 'updated_at': '2021-05-20T04:28:25Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:27.710320 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Checking router: 7b630b1e-4360-4f2e-9053-be7e631bfa3a for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:28:27.710677 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Updating router 7b630b1e-4360-4f2e-9053-be7e631bfa3a port 160f1898-b408-446e-8492-c4b331489205 binding host dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 20 04:28:27.756644 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:27.765604 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:27.927801 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 617c2db0-1597-44e9-9e82-8ea22dfd47a8 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:28:28.071741 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Allocated IP 172.20.0.1 (9798d104-d4de-413f-b41d-fb9d7e8a818e/c5423164-3d24-4562-871f-ab03914420dc/bce609e7-1912-4202-bc3c-9028a36c0adb) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:28:28.081166 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 617c2db0-1597-44e9-9e82-8ea22dfd47a8 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:28:28.081539 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:28.203590 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Notify callbacks [] for port, precommit_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:28.219105 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:28.235077 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.provisioning_blocks [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Transition to ACTIVE for port object bce609e7-1912-4202-bc3c-9028a36c0adb will not be triggered until provisioned by entity DHCP. {{(pid=91643) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:28:28.243283 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Transition to ACTIVE for port object 160f1898-b408-446e-8492-c4b331489205 will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:28:28.260929 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:28.261168 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:28.264739 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-85fc398b-da3e-408a-b1d0-b7838f862dae service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:28.267266 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:28.329807 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Network 9798d104-d4de-413f-b41d-fb9d7e8a818e is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:28:28.332756 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:28:28.337715 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:28:28.352687 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Attempting to bind port 160f1898-b408-446e-8492-c4b331489205 on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:28:28.353527 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Attempting to bind port 160f1898-b408-446e-8492-c4b331489205 by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:28:28.353989 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Attempting to bind port 160f1898-b408-446e-8492-c4b331489205 on network 712ce8c4-8feb-4719-bf13-e06992125cd9 {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:28:28.356375 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cf924b77-a1a6-4496-a46e-f71487656834 None None] Pushing event updated for resources: {'Port': ['ID=160f1898-b408-446e-8492-c4b331489205,revision_number=2']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:28.360249 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-cf924b77-a1a6-4496-a46e-f71487656834 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.093s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:28.363202 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 28, 15), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:28:28.363607 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Checking segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:28:28.364005 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:28.364395 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Bound using segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:28:28.365014 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Bound port: 160f1898-b408-446e-8492-c4b331489205, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:28:28.365231 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:28.375437 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Pushing event updated for resources: {'Port': ['ID=bce609e7-1912-4202-bc3c-9028a36c0adb,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:28.378318 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:28.400566 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-8003132'] for router_interface, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:28.439937 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] For port 160f1898-b408-446e-8492-c4b331489205, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:28:28.443075 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] For port 160f1898-b408-446e-8492-c4b331489205, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=160f1898-b408-446e-8492-c4b331489205,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:28:28.449262 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:28.453434 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:28.534986 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Routers {'7b630b1e-4360-4f2e-9053-be7e631bfa3a'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:28:28.535633 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:28:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['7b630b1e-4360-4f2e-9053-be7e631bfa3a'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:28.600041 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cf924b77-a1a6-4496-a46e-f71487656834 None None] Pushing event updated for resources: {'Port': ['ID=160f1898-b408-446e-8492-c4b331489205,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:28.604933 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-cf924b77-a1a6-4496-a46e-f71487656834 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.151s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:28.605523 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-9015fcca-62eb-4739-ae11-ceb24a51b3a1 None None] Provisioning for port bce609e7-1912-4202-bc3c-9028a36c0adb completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:28:28.621842 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-9015fcca-62eb-4739-ae11-ceb24a51b3a1 None None] Provisioning complete for port bce609e7-1912-4202-bc3c-9028a36c0adb triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:28:28.622219 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-9015fcca-62eb-4739-ae11-ceb24a51b3a1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:28.660633 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:28.745771 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-9015fcca-62eb-4739-ae11-ceb24a51b3a1 None None] Port bce609e7-1912-4202-bc3c-9028a36c0adb cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:28:28.766471 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] Set router 7b630b1e-4360-4f2e-9053-be7e631bfa3a gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:28:28.888403 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Notify callbacks [] for port, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:28.894178 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:28.920766 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9e93236a-7359-4953-a600-6b9e0d313161 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:28.999300 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-7924897', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-8003163'] for router_interface, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:29.002733 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Pushing event updated for resources: {'Port': ['ID=bce609e7-1912-4202-bc3c-9028a36c0adb,revision_number=2']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:29.006095 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.085s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:29.084559 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'name': 'tempest-TestMinimumBasicScenario-22498061-router', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158'}], 'enable_snat': True}, 'gw_port_id': '160f1898-b408-446e-8492-c4b331489205', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:24Z', 'updated_at': '2021-05-20T04:28:26Z', 'revision_number': 3, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'gw_port': {'id': '160f1898-b408-446e-8492-c4b331489205', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:79:e7:e5', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 544, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:25Z', 'updated_at': '2021-05-20T04:28:28Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:29.088544 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-676b559b-f2c1-4122-9fd2-b8655d485c5d None None] The sync data for ids ['7b630b1e-4360-4f2e-9053-be7e631bfa3a'] in dv-cinder-782405-9-iscsi is: [{'id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'name': 'tempest-TestMinimumBasicScenario-22498061-router', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158'}], 'enable_snat': True}, 'gw_port_id': '160f1898-b408-446e-8492-c4b331489205', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:24Z', 'updated_at': '2021-05-20T04:28:26Z', 'revision_number': 3, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'gw_port': {'id': '160f1898-b408-446e-8492-c4b331489205', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:79:e7:e5', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 544, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:25Z', 'updated_at': '2021-05-20T04:28:28Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:28:29.136982 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Network 9798d104-d4de-413f-b41d-fb9d7e8a818e is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:28:29.138178 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:28:29.338645 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Router 7b630b1e-4360-4f2e-9053-be7e631bfa3a has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91643) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:28:29.381504 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-b2cf116a-780f-4f08-a2d8-907868f0102c None None] Provisioning complete for port bce609e7-1912-4202-bc3c-9028a36c0adb triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:28:29.381887 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-b2cf116a-780f-4f08-a2d8-907868f0102c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:29.439813 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-b2cf116a-780f-4f08-a2d8-907868f0102c None None] Port bce609e7-1912-4202-bc3c-9028a36c0adb cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:28:29.634097 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:28:29.643447 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] 10.222.0.39 "PUT /v2.0/routers/7b630b1e-4360-4f2e-9053-be7e631bfa3a/add_router_interface HTTP/1.1" status: 200 len: 503 time: 2.5488400 May 20 04:28:29.959050 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 160f1898-b408-446e-8492-c4b331489205 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:28:30.030980 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port 160f1898-b408-446e-8492-c4b331489205 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:28:30.035635 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 160f1898-b408-446e-8492-c4b331489205 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:28:30.036125 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:30.155314 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:30.172242 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port 160f1898-b408-446e-8492-c4b331489205, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=160f1898-b408-446e-8492-c4b331489205,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:28:30.176906 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:30.178697 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:30.203488 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 41328) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:28:30.209215 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Request body: {'network': {'name': 'tempest-TestMinimumBasicScenario-1656849615-network', 'tenant_id': '5c8cd90c0a57413e889fa3808d4bdf52'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:28:30.213997 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:28:30.218321 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Usage tracker for resource:network and tenant:5c8cd90c0a57413e889fa3808d4bdf52 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:28:30.221316 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:28:30.227349 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:28:30.238517 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Made reservation on behalf of 5c8cd90c0a57413e889fa3808d4bdf52 for: {'network': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:28:30.240847 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-585852'] for network, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:30.244209 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=160f1898-b408-446e-8492-c4b331489205,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:30.247149 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.068s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:30.252885 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:30.267591 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:28:30.271495 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Usage tracker for resource:security_group_rule and tenant:5c8cd90c0a57413e889fa3808d4bdf52 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:28:30.274339 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:28:30.280354 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:28:30.363252 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:30.368571 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:30.371511 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-6422b44d-e479-410a-acab-89dfd0579690 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:30.399150 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 838} {{(pid=91643) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 20 04:28:30.407955 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=4fcfb45d-4ed7-4de4-a623-ae5bf3006001,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:30.411493 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.040s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:30.413967 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:30.414315 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.segments_db [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Added segment e3e8d115-0d77-4bf0-a504-52c3fb21936c of type vlan for network c4d0b06b-6db7-4af5-9ff8-304cb32a79cd May 20 04:28:30.440193 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8760688516720'] for network, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:30.471322 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915'] for network, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:30.472626 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:30.473207 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:30.476410 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:30.574798 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Pushing event updated for resources: {'Network': ['ID=c4d0b06b-6db7-4af5-9ff8-304cb32a79cd,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:30.578933 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] 10.222.0.39 "POST /v2.0/networks HTTP/1.1" status: 201 len: 888 time: 0.3731802 May 20 04:28:30.578933 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.102s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:30.580541 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 41332) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:28:30.586417 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-ac5c230b-aa49-48f8-96ae-fc88dd6d809a admin admin] Request body: {'subnet': {'network_id': 'c4d0b06b-6db7-4af5-9ff8-304cb32a79cd', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestMinimumBasicScenario-1656849615-subnet', 'tenant_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'ip_version': 4}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:28:30.587015 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Network c4d0b06b-6db7-4af5-9ff8-304cb32a79cd details requested by agent ovs-agent-dv-cinder-782405-9-iscsi with host dv-cinder-782405-9-iscsi {{(pid=91645) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 20 04:28:30.591456 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-ac5c230b-aa49-48f8-96ae-fc88dd6d809a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:28:30.595760 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-ac5c230b-aa49-48f8-96ae-fc88dd6d809a admin admin] Usage tracker for resource:subnet and tenant:5c8cd90c0a57413e889fa3808d4bdf52 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:28:30.598609 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-ac5c230b-aa49-48f8-96ae-fc88dd6d809a admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:28:30.604685 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-ac5c230b-aa49-48f8-96ae-fc88dd6d809a admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:28:30.621247 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-ac5c230b-aa49-48f8-96ae-fc88dd6d809a admin admin] Made reservation on behalf of 5c8cd90c0a57413e889fa3808d4bdf52 for: {'subnet': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:28:30.624470 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-ac5c230b-aa49-48f8-96ae-fc88dd6d809a admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2172529'] for subnet, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:30.631355 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Sync routers for ids ['7b630b1e-4360-4f2e-9053-be7e631bfa3a'] in dv-cinder-782405-9-iscsi {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:28:30.641453 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Routers {'7b630b1e-4360-4f2e-9053-be7e631bfa3a'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:28:30.642116 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:28:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['7b630b1e-4360-4f2e-9053-be7e631bfa3a'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:30.819140 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Set router 7b630b1e-4360-4f2e-9053-be7e631bfa3a gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:28:31.006649 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-ac5c230b-aa49-48f8-96ae-fc88dd6d809a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-2050444', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2172483'] for subnet, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:31.010410 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-1e204bf6-94c1-450b-a254-0641f88ad1c5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:31.080879 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ac5c230b-aa49-48f8-96ae-fc88dd6d809a admin admin] Pushing event updated for resources: {'Subnet': ['ID=ce429852-561a-4890-b1d6-22338214dd01,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:31.086729 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-ac5c230b-aa49-48f8-96ae-fc88dd6d809a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.076s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:31.093087 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ac5c230b-aa49-48f8-96ae-fc88dd6d809a admin admin] Checking segment: {'id': 'e3e8d115-0d77-4bf0-a504-52c3fb21936c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 838, 'network_id': 'c4d0b06b-6db7-4af5-9ff8-304cb32a79cd'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91643) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:28:31.093439 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-ac5c230b-aa49-48f8-96ae-fc88dd6d809a admin admin] Network hostable DHCP agents. Network: c4d0b06b-6db7-4af5-9ff8-304cb32a79cd, hostable agents: ['fffae25a-f450-4182-8f9e-ac29d143f6c9'], hosted agents: [] {{(pid=91643) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 20 04:28:31.111704 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-ac5c230b-aa49-48f8-96ae-fc88dd6d809a admin admin] Network c4d0b06b-6db7-4af5-9ff8-304cb32a79cd is scheduled to be hosted by DHCP agent fffae25a-f450-4182-8f9e-ac29d143f6c9 with binding_index 1 {{(pid=91643) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 20 04:28:31.122364 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-ac5c230b-aa49-48f8-96ae-fc88dd6d809a admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-782405-9-iscsi, c4d0b06b-6db7-4af5-9ff8-304cb32a79cd) {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:28:31.130733 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-ac5c230b-aa49-48f8-96ae-fc88dd6d809a admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:31.131339 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-ac5c230b-aa49-48f8-96ae-fc88dd6d809a admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:31.135480 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'name': 'tempest-TestMinimumBasicScenario-22498061-router', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158'}], 'enable_snat': True}, 'gw_port_id': '160f1898-b408-446e-8492-c4b331489205', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:24Z', 'updated_at': '2021-05-20T04:28:28Z', 'revision_number': 4, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'gw_port': {'id': '160f1898-b408-446e-8492-c4b331489205', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:79:e7:e5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 544, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:25Z', 'updated_at': '2021-05-20T04:28:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'bce609e7-1912-4202-bc3c-9028a36c0adb', 'name': '', 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'mac_address': 'fa:16:3e:57:64:c1', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_interface', 'standard_attr_id': 545, 'fixed_ips': [{'subnet_id': 'c5423164-3d24-4562-871f-ab03914420dc', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:27Z', 'updated_at': '2021-05-20T04:28:28Z', 'revision_number': 2, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'subnets': [{'id': 'c5423164-3d24-4562-871f-ab03914420dc', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:31.139993 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-438cef4f-a5e6-4db6-9ed3-ebc97874d396 None None] Network c4d0b06b-6db7-4af5-9ff8-304cb32a79cd requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:28:31.140493 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Checking router: 7b630b1e-4360-4f2e-9053-be7e631bfa3a for host: dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:28:31.140683 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Updating router 7b630b1e-4360-4f2e-9053-be7e631bfa3a port bce609e7-1912-4202-bc3c-9028a36c0adb binding host dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 20 04:28:31.154894 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-ac5c230b-aa49-48f8-96ae-fc88dd6d809a admin admin] 10.222.0.39 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 851 time: 0.5732236 May 20 04:28:31.157531 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 41338) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:28:31.163704 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Request body: {'router': {'name': 'tempest-TestMinimumBasicScenario-1656849615-router', 'tenant_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:28:31.169919 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:28:31.175429 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Usage tracker for resource:router and tenant:5c8cd90c0a57413e889fa3808d4bdf52 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:28:31.178666 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:28:31.184978 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:28:31.194048 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:31.197934 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-8071540', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-260712'] for router, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:31.199030 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-8056436', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-8053301', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-7881139', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-260707'] for router, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:31.214255 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:31.367645 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcp85c0b5c8-c799-5455-967d-3e9c2c8b4839-c4d0b06b-6db7-4af5-9ff8-304cb32a79cd', 'network_id': 'c4d0b06b-6db7-4af5-9ff8-304cb32a79cd', 'tenant_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'fixed_ips': [{'subnet_id': 'ce429852-561a-4890-b1d6-22338214dd01'}]}} from dv-cinder-782405-9-iscsi. {{(pid=91645) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 20 04:28:31.368578 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:31.482192 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-8053308', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-8071430', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-260685'] for router, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:31.524097 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.db_base_plugin_common [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Allocated IP 172.20.0.2 (c4d0b06b-6db7-4af5-9ff8-304cb32a79cd/ce429852-561a-4890-b1d6-22338214dd01/0d0ad457-9322-4fb5-abac-feaec6951e5a) {{(pid=91645) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:28:31.538781 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Notify callbacks [] for port, precommit_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:31.550697 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.provisioning_blocks [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Transition to ACTIVE for port object bce609e7-1912-4202-bc3c-9028a36c0adb will not be triggered until provisioned by entity L2. {{(pid=91646) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:28:31.557451 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:31.561262 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:31.628481 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a6c6f9ac-d3a6-458b-8d43-939ec50454d4 None None] Pushing event updated for resources: {'Port': ['ID=bce609e7-1912-4202-bc3c-9028a36c0adb,revision_number=3']} {{(pid=91646) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:31.751319 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Notify callbacks [] for port, precommit_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:31.763474 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Transition to ACTIVE for port object 0d0ad457-9322-4fb5-abac-feaec6951e5a will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:28:31.782776 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Transition to ACTIVE for port object 0d0ad457-9322-4fb5-abac-feaec6951e5a will not be triggered until provisioned by entity DHCP. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:28:31.790421 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399'] for port, after_create {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:31.795959 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-ace8ac61-9ace-48af-93fb-1ea478ec7cb2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:31.809821 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6c6f9ac-d3a6-458b-8d43-939ec50454d4 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:31.810305 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6c6f9ac-d3a6-458b-8d43-939ec50454d4 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:31.810488 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6c6f9ac-d3a6-458b-8d43-939ec50454d4 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:31.810799 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6c6f9ac-d3a6-458b-8d43-939ec50454d4 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:31.811167 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6c6f9ac-d3a6-458b-8d43-939ec50454d4 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:31.811576 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6c6f9ac-d3a6-458b-8d43-939ec50454d4 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:31.811886 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6c6f9ac-d3a6-458b-8d43-939ec50454d4 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:31.812208 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6c6f9ac-d3a6-458b-8d43-939ec50454d4 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:31.812542 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6c6f9ac-d3a6-458b-8d43-939ec50454d4 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:31.812859 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6c6f9ac-d3a6-458b-8d43-939ec50454d4 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:31.813171 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6c6f9ac-d3a6-458b-8d43-939ec50454d4 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:31.813477 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6c6f9ac-d3a6-458b-8d43-939ec50454d4 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91646) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:31.818334 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Attempting to bind port bce609e7-1912-4202-bc3c-9028a36c0adb on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:28:31.818921 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Attempting to bind port bce609e7-1912-4202-bc3c-9028a36c0adb by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': 'd558b057-1c72-4efe-ab53-567b572e2de5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 801, 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e'}] {{(pid=91646) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:28:31.819629 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Attempting to bind port bce609e7-1912-4202-bc3c-9028a36c0adb on network 9798d104-d4de-413f-b41d-fb9d7e8a818e {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:28:31.822293 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-a6c6f9ac-d3a6-458b-8d43-939ec50454d4 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.261s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:31.827425 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 28, 15), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:28:31.827733 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Checking segment: {'id': 'd558b057-1c72-4efe-ab53-567b572e2de5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 801, 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91646) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:28:31.828092 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:31.828479 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Bound using segment: {'id': 'd558b057-1c72-4efe-ab53-567b572e2de5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 801, 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e'} {{(pid=91646) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:28:31.828993 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.managers [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Bound port: bce609e7-1912-4202-bc3c-9028a36c0adb, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'd558b057-1c72-4efe-ab53-567b572e2de5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 801, 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e'}}] {{(pid=91646) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:28:31.829402 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:31.864737 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Network c4d0b06b-6db7-4af5-9ff8-304cb32a79cd is already hosted by enough agents. {{(pid=91645) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:28:31.871426 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.scheduler.base_scheduler [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Resources bound (agent type, host, resource id): {{(pid=91645) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:28:31.874271 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Received port after_create {{(pid=91645) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:28:31.919037 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Pushing event updated for resources: {'Port': ['ID=0d0ad457-9322-4fb5-abac-feaec6951e5a,revision_number=1']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:31.922123 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.126s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:31.923590 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:31.956072 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.db [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] For port bce609e7-1912-4202-bc3c-9028a36c0adb, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91646) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:28:31.960110 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.plugins.ml2.db [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] For port bce609e7-1912-4202-bc3c-9028a36c0adb, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=bce609e7-1912-4202-bc3c-9028a36c0adb,segment=NetworkSegment(d558b057-1c72-4efe-ab53-567b572e2de5),segment_id=d558b057-1c72-4efe-ab53-567b572e2de5)] {{(pid=91646) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:28:31.963715 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:31.966506 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron_lib.callbacks.manager [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91646) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:31.969631 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 160f1898-b408-446e-8492-c4b331489205 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:28:31.972172 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:32.046929 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Routers {'7b630b1e-4360-4f2e-9053-be7e631bfa3a'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:28:32.047583 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:28:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['7b630b1e-4360-4f2e-9053-be7e631bfa3a'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:32.052876 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Attempting to bind port 0d0ad457-9322-4fb5-abac-feaec6951e5a on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:28:32.053253 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Attempting to bind port 0d0ad457-9322-4fb5-abac-feaec6951e5a by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': 'e3e8d115-0d77-4bf0-a504-52c3fb21936c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 838, 'network_id': 'c4d0b06b-6db7-4af5-9ff8-304cb32a79cd'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:28:32.053619 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Attempting to bind port 0d0ad457-9322-4fb5-abac-feaec6951e5a on network c4d0b06b-6db7-4af5-9ff8-304cb32a79cd {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:28:32.061325 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 28, 15), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:28:32.061706 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Checking segment: {'id': 'e3e8d115-0d77-4bf0-a504-52c3fb21936c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 838, 'network_id': 'c4d0b06b-6db7-4af5-9ff8-304cb32a79cd'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:28:32.062112 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:32.062504 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Bound using segment: {'id': 'e3e8d115-0d77-4bf0-a504-52c3fb21936c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 838, 'network_id': 'c4d0b06b-6db7-4af5-9ff8-304cb32a79cd'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:28:32.062977 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Bound port: 0d0ad457-9322-4fb5-abac-feaec6951e5a, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'e3e8d115-0d77-4bf0-a504-52c3fb21936c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 838, 'network_id': 'c4d0b06b-6db7-4af5-9ff8-304cb32a79cd'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:28:32.063383 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:32.089893 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 160f1898-b408-446e-8492-c4b331489205 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:28:32.090315 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:32.108635 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Allocated IP 172.27.0.119 (712ce8c4-8feb-4719-bf13-e06992125cd9/e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5/a1e201be-75e9-4fbb-b3c4-47eeb5b0221c) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:28:32.113958 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a6c6f9ac-d3a6-458b-8d43-939ec50454d4 None None] Pushing event updated for resources: {'Port': ['ID=bce609e7-1912-4202-bc3c-9028a36c0adb,revision_number=4']} {{(pid=91646) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:32.129444 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG oslo_concurrency.lockutils [None req-a6c6f9ac-d3a6-458b-8d43-939ec50454d4 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.153s {{(pid=91646) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:32.260415 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] For port 0d0ad457-9322-4fb5-abac-feaec6951e5a, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:28:32.266121 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] For port 0d0ad457-9322-4fb5-abac-feaec6951e5a, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=0d0ad457-9322-4fb5-abac-feaec6951e5a,segment=NetworkSegment(e3e8d115-0d77-4bf0-a504-52c3fb21936c),segment_id=e3e8d115-0d77-4bf0-a504-52c3fb21936c)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:28:32.269490 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] Set router 7b630b1e-4360-4f2e-9053-be7e631bfa3a gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:28:32.290285 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:32.320199 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:32.321638 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:32.325973 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:32.326612 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-c5d3b055-34ea-49b5-a864-db1e10f6eeb2 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:32.399788 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:28:32.402695 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:28:32.406039 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:28:32.425495 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Pushing event updated for resources: {'Port': ['ID=0d0ad457-9322-4fb5-abac-feaec6951e5a,revision_number=2']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:32.439573 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:32.445304 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Pushing event updated for resources: {'Port': ['ID=a1e201be-75e9-4fbb-b3c4-47eeb5b0221c,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:32.456094 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:32.456822 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:32.457338 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:32.457849 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:32.458363 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:32.458872 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:32.459673 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:32.460186 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:32.460683 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:32.461246 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:32.461810 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:32.462363 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:28:32.467257 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.140s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:32.651576 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'name': 'tempest-TestMinimumBasicScenario-22498061-router', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158'}], 'enable_snat': True}, 'gw_port_id': '160f1898-b408-446e-8492-c4b331489205', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:24Z', 'updated_at': '2021-05-20T04:28:28Z', 'revision_number': 4, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'gw_port': {'id': '160f1898-b408-446e-8492-c4b331489205', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:79:e7:e5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 544, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:25Z', 'updated_at': '2021-05-20T04:28:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'bce609e7-1912-4202-bc3c-9028a36c0adb', 'name': '', 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'mac_address': 'fa:16:3e:57:64:c1', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_interface', 'standard_attr_id': 545, 'fixed_ips': [{'subnet_id': 'c5423164-3d24-4562-871f-ab03914420dc', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:27Z', 'updated_at': '2021-05-20T04:28:31Z', 'revision_number': 4, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'subnets': [{'id': 'c5423164-3d24-4562-871f-ab03914420dc', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:32.656736 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0adeef18-ca75-44d6-9964-a44081793f1d None None] The sync data for ids ['7b630b1e-4360-4f2e-9053-be7e631bfa3a'] in dv-cinder-782405-9-iscsi is: [{'id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'name': 'tempest-TestMinimumBasicScenario-22498061-router', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158'}], 'enable_snat': True}, 'gw_port_id': '160f1898-b408-446e-8492-c4b331489205', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:24Z', 'updated_at': '2021-05-20T04:28:28Z', 'revision_number': 4, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'gw_port': {'id': '160f1898-b408-446e-8492-c4b331489205', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:79:e7:e5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 544, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:25Z', 'updated_at': '2021-05-20T04:28:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'bce609e7-1912-4202-bc3c-9028a36c0adb', 'name': '', 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'mac_address': 'fa:16:3e:57:64:c1', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_interface', 'standard_attr_id': 545, 'fixed_ips': [{'subnet_id': 'c5423164-3d24-4562-871f-ab03914420dc', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:27Z', 'updated_at': '2021-05-20T04:28:31Z', 'revision_number': 4, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'subnets': [{'id': 'c5423164-3d24-4562-871f-ab03914420dc', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:28:32.807523 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8db69625-285e-4102-a404-4ee4ca7e96de req-7d85aca2-443e-4e8f-8569-e5740793a31b tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:32.813621 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8db69625-285e-4102-a404-4ee4ca7e96de req-7d85aca2-443e-4e8f-8569-e5740793a31b tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 945 time: 0.1529095 May 20 04:28:32.875302 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8db69625-285e-4102-a404-4ee4ca7e96de req-a037e932-c3df-4ea5-ba32-335d118fdc60 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0501354 May 20 04:28:32.967637 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:33.318708 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-8071551', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-8053391'] for router, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:33.485220 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-248233ff-b801-44d7-a46e-32e3facf8fae None None] Network c4d0b06b-6db7-4af5-9ff8-304cb32a79cd requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:28:33.596377 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-d64aaf96-1304-4002-8712-c6ea2a6943db None None] Provisioning for port 0d0ad457-9322-4fb5-abac-feaec6951e5a completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:28:33.646554 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Router 007d36bf-1482-45ff-a392-0fb4d38879a7 is scheduled to L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 with binding_index 1 {{(pid=91644) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 20 04:28:33.828956 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:28:33.837889 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Sync routers for ids ['007d36bf-1482-45ff-a392-0fb4d38879a7'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:28:33.848058 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Routers {'007d36bf-1482-45ff-a392-0fb4d38879a7'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:28:33.848661 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:28:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['007d36bf-1482-45ff-a392-0fb4d38879a7'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:33.856785 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Notify callbacks [] for router, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:33.860735 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] 10.222.0.39 "POST /v2.0/routers HTTP/1.1" status: 201 len: 927 time: 2.6989481 May 20 04:28:33.861104 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 41346) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:28:34.056064 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device bce609e7-1912-4202-bc3c-9028a36c0adb up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:28:34.160058 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Set router 007d36bf-1482-45ff-a392-0fb4d38879a7 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:28:34.265106 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05fad095-11f5-4985-8d23-6171241f3de8 req-3250995b-201d-4d0a-a8d5-1ebe275318c8 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:34.266709 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-05fad095-11f5-4985-8d23-6171241f3de8 req-3250995b-201d-4d0a-a8d5-1ebe275318c8 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 945 time: 0.2341051 May 20 04:28:34.282179 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port bce609e7-1912-4202-bc3c-9028a36c0adb completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:28:34.300180 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port bce609e7-1912-4202-bc3c-9028a36c0adb triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:28:34.300560 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:34.444418 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-05fad095-11f5-4985-8d23-6171241f3de8 req-c98ce0d3-a451-4f3e-aa22-6927a1b49948 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1690552 May 20 04:28:34.533933 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:34.575395 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port bce609e7-1912-4202-bc3c-9028a36c0adb, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=bce609e7-1912-4202-bc3c-9028a36c0adb,segment=NetworkSegment(d558b057-1c72-4efe-ab53-567b572e2de5),segment_id=d558b057-1c72-4efe-ab53-567b572e2de5)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:28:34.580473 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '007d36bf-1482-45ff-a392-0fb4d38879a7', 'name': 'tempest-TestMinimumBasicScenario-1656849615-router', 'tenant_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.119'}], 'enable_snat': True}, 'gw_port_id': 'a1e201be-75e9-4fbb-b3c4-47eeb5b0221c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:31Z', 'updated_at': '2021-05-20T04:28:32Z', 'revision_number': 3, 'project_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'gw_port': {'id': 'a1e201be-75e9-4fbb-b3c4-47eeb5b0221c', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6e:9b:ea', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '007d36bf-1482-45ff-a392-0fb4d38879a7', 'device_owner': 'network:router_gateway', 'standard_attr_id': 556, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.119', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:31Z', 'updated_at': '2021-05-20T04:28:32Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:34.584313 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:34.585154 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 0d0ad457-9322-4fb5-abac-feaec6951e5a up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:28:34.609739 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:34.646235 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Checking router: 007d36bf-1482-45ff-a392-0fb4d38879a7 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:28:34.646592 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Updating router 007d36bf-1482-45ff-a392-0fb4d38879a7 port a1e201be-75e9-4fbb-b3c4-47eeb5b0221c binding host dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 20 04:28:34.722867 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a999d49f-6d2b-43ea-8ec7-711ea132e6d9 req-efbd5b38-a4fb-4053-b5e7-cdb770715bb8 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:34.724240 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a999d49f-6d2b-43ea-8ec7-711ea132e6d9 req-efbd5b38-a4fb-4053-b5e7-cdb770715bb8 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.74 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 945 time: 0.3787792 May 20 04:28:34.745876 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:34.772832 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=bce609e7-1912-4202-bc3c-9028a36c0adb,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:34.776005 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.166s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:34.851634 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port 0d0ad457-9322-4fb5-abac-feaec6951e5a completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:28:34.864516 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 0d0ad457-9322-4fb5-abac-feaec6951e5a triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:28:34.865027 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:34.995603 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-a999d49f-6d2b-43ea-8ec7-711ea132e6d9 req-15abce54-41b7-4bbf-87a2-b6e674f4f370 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Deleting port 4a6aa217-4367-4b88-9233-0a8aeee2352d {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:28:34.996229 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-a999d49f-6d2b-43ea-8ec7-711ea132e6d9 req-15abce54-41b7-4bbf-87a2-b6e674f4f370 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:35.031033 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:35.144948 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:35.208479 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-a999d49f-6d2b-43ea-8ec7-711ea132e6d9 req-15abce54-41b7-4bbf-87a2-b6e674f4f370 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] For port 4a6aa217-4367-4b88-9233-0a8aeee2352d, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=4a6aa217-4367-4b88-9233-0a8aeee2352d,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:28:35.208823 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-a999d49f-6d2b-43ea-8ec7-711ea132e6d9 req-15abce54-41b7-4bbf-87a2-b6e674f4f370 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:35.219583 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port 0d0ad457-9322-4fb5-abac-feaec6951e5a, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=0d0ad457-9322-4fb5-abac-feaec6951e5a,segment=NetworkSegment(e3e8d115-0d77-4bf0-a504-52c3fb21936c),segment_id=e3e8d115-0d77-4bf0-a504-52c3fb21936c)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:28:35.231095 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:35.235330 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-49b55c97-b696-4cc4-a467-619b81859ef7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:35.308781 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Notify callbacks [] for port, precommit_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:35.330601 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Transition to ACTIVE for port object a1e201be-75e9-4fbb-b3c4-47eeb5b0221c will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:28:35.333070 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=0d0ad457-9322-4fb5-abac-feaec6951e5a,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:35.336955 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.102s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:35.340470 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:35.346194 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:35.420564 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Attempting to bind port a1e201be-75e9-4fbb-b3c4-47eeb5b0221c on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:28:35.420954 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Attempting to bind port a1e201be-75e9-4fbb-b3c4-47eeb5b0221c by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:28:35.421349 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Attempting to bind port a1e201be-75e9-4fbb-b3c4-47eeb5b0221c on network 712ce8c4-8feb-4719-bf13-e06992125cd9 {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:28:35.423573 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-925dbba8-f38d-4e26-af1f-c06367423c42 None None] Pushing event updated for resources: {'Port': ['ID=a1e201be-75e9-4fbb-b3c4-47eeb5b0221c,revision_number=2']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:35.427180 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-925dbba8-f38d-4e26-af1f-c06367423c42 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:35.429824 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 28, 15), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:28:35.430215 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Checking segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:28:35.430618 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:35.431069 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Bound using segment: {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:28:35.431485 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Bound port: a1e201be-75e9-4fbb-b3c4-47eeb5b0221c, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '240fc062-60cd-4ee1-8e1d-b9956ca52b7b', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:28:35.431859 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:35.497594 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Allocated IP 172.20.0.1 (c4d0b06b-6db7-4af5-9ff8-304cb32a79cd/ce429852-561a-4890-b1d6-22338214dd01/8b38f061-73b2-48b5-bd6f-a15baf789252) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:28:35.508231 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] For port a1e201be-75e9-4fbb-b3c4-47eeb5b0221c, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:28:35.512269 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] For port a1e201be-75e9-4fbb-b3c4-47eeb5b0221c, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=a1e201be-75e9-4fbb-b3c4-47eeb5b0221c,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:28:35.517793 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:35.522582 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:35.610921 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Routers {'007d36bf-1482-45ff-a392-0fb4d38879a7'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:28:35.611675 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:28:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['007d36bf-1482-45ff-a392-0fb4d38879a7'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:35.676723 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:35.684646 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-925dbba8-f38d-4e26-af1f-c06367423c42 None None] Pushing event updated for resources: {'Port': ['ID=a1e201be-75e9-4fbb-b3c4-47eeb5b0221c,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:35.688029 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-925dbba8-f38d-4e26-af1f-c06367423c42 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.165s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:35.748577 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-a999d49f-6d2b-43ea-8ec7-711ea132e6d9 req-15abce54-41b7-4bbf-87a2-b6e674f4f370 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for floatingip, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:35.748846 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-a999d49f-6d2b-43ea-8ec7-711ea132e6d9 req-15abce54-41b7-4bbf-87a2-b6e674f4f370 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-8003251'] for floatingip, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:35.749151 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.l3_db [req-a999d49f-6d2b-43ea-8ec7-711ea132e6d9 req-15abce54-41b7-4bbf-87a2-b6e674f4f370 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Floating IP d2b40541-a315-4701-99bd-be17179c9b8b disassociated. External IP: 172.27.0.245, port: 4a6aa217-4367-4b88-9233-0a8aeee2352d. May 20 04:28:35.749483 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-a999d49f-6d2b-43ea-8ec7-711ea132e6d9 req-15abce54-41b7-4bbf-87a2-b6e674f4f370 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Calling delete_port for 4a6aa217-4367-4b88-9233-0a8aeee2352d owned by compute:nova {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:28:35.807779 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] Set router 007d36bf-1482-45ff-a392-0fb4d38879a7 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:28:35.837192 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b21f4237-3587-47f4-bce1-7fc933c8933f req-6d30d959-24f2-4e84-a9f3-cab444c63258 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:35.838636 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b21f4237-3587-47f4-bce1-7fc933c8933f req-6d30d959-24f2-4e84-a9f3-cab444c63258 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 945 time: 0.2774420 May 20 04:28:35.849945 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:35.901946 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.provisioning_blocks [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Transition to ACTIVE for port object 8b38f061-73b2-48b5-bd6f-a15baf789252 will not be triggered until provisioned by entity DHCP. {{(pid=91643) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:28:35.946463 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b21f4237-3587-47f4-bce1-7fc933c8933f req-1e091173-296f-4e89-807e-d14bbe4b0758 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1001894 May 20 04:28:35.985956 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-a999d49f-6d2b-43ea-8ec7-711ea132e6d9 req-15abce54-41b7-4bbf-87a2-b6e674f4f370 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:35.990709 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device bce609e7-1912-4202-bc3c-9028a36c0adb up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:28:35.992026 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:36.037868 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:36.043724 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-15abce54-41b7-4bbf-87a2-b6e674f4f370 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event deleted for resources: {'Port': ['ID=4a6aa217-4367-4b88-9233-0a8aeee2352d,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:36.049774 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-15abce54-41b7-4bbf-87a2-b6e674f4f370 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.058s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:36.050266 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-01ec7cfe-f64e-42bc-84c0-7fb02370e249 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.007s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:36.132788 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port bce609e7-1912-4202-bc3c-9028a36c0adb triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:28:36.133230 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:36.243730 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '007d36bf-1482-45ff-a392-0fb4d38879a7', 'name': 'tempest-TestMinimumBasicScenario-1656849615-router', 'tenant_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.119'}], 'enable_snat': True}, 'gw_port_id': 'a1e201be-75e9-4fbb-b3c4-47eeb5b0221c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:31Z', 'updated_at': '2021-05-20T04:28:32Z', 'revision_number': 3, 'project_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'gw_port': {'id': 'a1e201be-75e9-4fbb-b3c4-47eeb5b0221c', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6e:9b:ea', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '007d36bf-1482-45ff-a392-0fb4d38879a7', 'device_owner': 'network:router_gateway', 'standard_attr_id': 556, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.119', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:31Z', 'updated_at': '2021-05-20T04:28:35Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:36.265711 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4938b869-e55c-4bf4-8fa8-d7affe311678 None None] The sync data for ids ['007d36bf-1482-45ff-a392-0fb4d38879a7'] in dv-cinder-782405-9-iscsi is: [{'id': '007d36bf-1482-45ff-a392-0fb4d38879a7', 'name': 'tempest-TestMinimumBasicScenario-1656849615-router', 'tenant_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.119'}], 'enable_snat': True}, 'gw_port_id': 'a1e201be-75e9-4fbb-b3c4-47eeb5b0221c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:31Z', 'updated_at': '2021-05-20T04:28:32Z', 'revision_number': 3, 'project_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'gw_port': {'id': 'a1e201be-75e9-4fbb-b3c4-47eeb5b0221c', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6e:9b:ea', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '007d36bf-1482-45ff-a392-0fb4d38879a7', 'device_owner': 'network:router_gateway', 'standard_attr_id': 556, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.119', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:31Z', 'updated_at': '2021-05-20T04:28:35Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:28:36.276895 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Network c4d0b06b-6db7-4af5-9ff8-304cb32a79cd is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:28:36.278056 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 0d0ad457-9322-4fb5-abac-feaec6951e5a up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:28:36.279258 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:28:36.293191 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:28:36.358640 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 0d0ad457-9322-4fb5-abac-feaec6951e5a triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:28:36.359249 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:36.362009 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Pushing event updated for resources: {'Port': ['ID=8b38f061-73b2-48b5-bd6f-a15baf789252,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:36.373692 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.323s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:36.430217 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-8003132'] for router_interface, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:36.498658 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [req-a999d49f-6d2b-43ea-8ec7-711ea132e6d9 req-15abce54-41b7-4bbf-87a2-b6e674f4f370 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Router 4969b258-f606-4def-b666-09764c475749 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91643) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:28:36.596715 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-2e231de9-ead4-418d-9f14-8e5a1114e66c None None] Provisioning for port 8b38f061-73b2-48b5-bd6f-a15baf789252 completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:28:36.601447 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-2e231de9-ead4-418d-9f14-8e5a1114e66c None None] Provisioning complete for port 8b38f061-73b2-48b5-bd6f-a15baf789252 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:28:36.601822 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-2e231de9-ead4-418d-9f14-8e5a1114e66c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:36.667469 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-2e231de9-ead4-418d-9f14-8e5a1114e66c None None] Port 8b38f061-73b2-48b5-bd6f-a15baf789252 cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:28:36.788244 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-a999d49f-6d2b-43ea-8ec7-711ea132e6d9 req-15abce54-41b7-4bbf-87a2-b6e674f4f370 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:28:36.799106 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9f74dc2f-10bd-470e-960e-2406160b1bd4 None None] Sync routers for ids ['4969b258-f606-4def-b666-09764c475749'] in dv-cinder-782405-9-iscsi {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:28:36.809766 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9f74dc2f-10bd-470e-960e-2406160b1bd4 None None] Routers {'4969b258-f606-4def-b666-09764c475749'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:28:36.810585 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-9f74dc2f-10bd-470e-960e-2406160b1bd4 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:28:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['4969b258-f606-4def-b666-09764c475749'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:36.868371 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:36.884581 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [req-a999d49f-6d2b-43ea-8ec7-711ea132e6d9 req-15abce54-41b7-4bbf-87a2-b6e674f4f370 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] No DVR routers for this DVR port 4a6aa217-4367-4b88-9233-0a8aeee2352d on host hv-cinder-78240 {{(pid=91643) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:28:36.889263 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-a999d49f-6d2b-43ea-8ec7-711ea132e6d9 req-15abce54-41b7-4bbf-87a2-b6e674f4f370 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:36.892135 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71', 'name': 'network-vif-deleted', 'tag': '4a6aa217-4367-4b88-9233-0a8aeee2352d'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:28:36.934285 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a999d49f-6d2b-43ea-8ec7-711ea132e6d9 req-15abce54-41b7-4bbf-87a2-b6e674f4f370 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.74 "DELETE /v2.0/ports/4a6aa217-4367-4b88-9233-0a8aeee2352d HTTP/1.1" status: 204 len: 173 time: 2.2029576 May 20 04:28:36.964211 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71', 'name': 'network-vif-deleted', 'tag': '4a6aa217-4367-4b88-9233-0a8aeee2352d', 'status': 'completed', 'code': 200} May 20 04:28:37.011330 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-9f74dc2f-10bd-470e-960e-2406160b1bd4 None None] Set router 4969b258-f606-4def-b666-09764c475749 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:28:37.236501 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-e032ef64-90e3-4001-8191-e59c1a7730e6 req-ed957807-8521-454a-882a-614db7f9d013 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Synchronizing usage tracker for tenant:0e3baa0ea24d42f3ab59e40845b350fd on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:28:37.240614 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Notify callbacks [] for port, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:37.256855 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:37.326588 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-c91dd04b-e52f-464b-abde-e930faac1453 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:37.352478 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-9f74dc2f-10bd-470e-960e-2406160b1bd4 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:50Z', 'revision_number': 5, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:37.357064 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9f74dc2f-10bd-470e-960e-2406160b1bd4 None None] Checking router: 4969b258-f606-4def-b666-09764c475749 for host: dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:28:37.365677 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9f74dc2f-10bd-470e-960e-2406160b1bd4 None None] Routers {'4969b258-f606-4def-b666-09764c475749'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:28:37.366479 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-9f74dc2f-10bd-470e-960e-2406160b1bd4 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:28:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['4969b258-f606-4def-b666-09764c475749'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:37.377367 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-e032ef64-90e3-4001-8191-e59c1a7730e6 req-ed957807-8521-454a-882a-614db7f9d013 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Unset dirty status for tenant:0e3baa0ea24d42f3ab59e40845b350fd on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:28:37.378658 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e032ef64-90e3-4001-8191-e59c1a7730e6 req-ed957807-8521-454a-882a-614db7f9d013 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=bb43e0fb-fb4e-44f6-86f2-6825c2bfcb71 HTTP/1.1" status: 200 len: 210 time: 0.1964281 May 20 04:28:37.412732 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Pushing event updated for resources: {'Port': ['ID=8b38f061-73b2-48b5-bd6f-a15baf789252,revision_number=2']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:37.416315 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.090s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:37.424322 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-7924897', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-8003163'] for router_interface, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:37.553055 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-9f74dc2f-10bd-470e-960e-2406160b1bd4 None None] Set router 4969b258-f606-4def-b666-09764c475749 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:28:37.573103 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Network c4d0b06b-6db7-4af5-9ff8-304cb32a79cd is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:28:37.574403 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:28:37.773288 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Router 007d36bf-1482-45ff-a392-0fb4d38879a7 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91643) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:28:37.784264 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 4a6aa217-4367-4b88-9233-0a8aeee2352d no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:28:37.855774 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: 4a6aa217-4367-4b88-9233-0a8aeee2352d {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:28:37.858259 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 4a6aa217-4367-4b88-9233-0a8aeee2352d not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:28:37.903172 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-21842199-7bbe-4a5d-9dde-280182ef90bd None None] Provisioning complete for port 8b38f061-73b2-48b5-bd6f-a15baf789252 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:28:37.903576 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-21842199-7bbe-4a5d-9dde-280182ef90bd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:37.948293 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-9f74dc2f-10bd-470e-960e-2406160b1bd4 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:50Z', 'revision_number': 5, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:37.960270 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9f74dc2f-10bd-470e-960e-2406160b1bd4 None None] The sync data for ids ['4969b258-f606-4def-b666-09764c475749'] in dv-cinder-782405-9-iscsi is: [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:50Z', 'revision_number': 5, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:28:38.007728 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-21842199-7bbe-4a5d-9dde-280182ef90bd None None] Port 8b38f061-73b2-48b5-bd6f-a15baf789252 cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:28:38.035805 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device a1e201be-75e9-4fbb-b3c4-47eeb5b0221c up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:28:38.120465 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port a1e201be-75e9-4fbb-b3c4-47eeb5b0221c completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:28:38.125913 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port a1e201be-75e9-4fbb-b3c4-47eeb5b0221c triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:28:38.126422 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:38.199271 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:28:38.216326 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-9fc71418-86ec-4f11-aab2-318a4a8c63f5 admin admin] 10.222.0.39 "PUT /v2.0/routers/007d36bf-1482-45ff-a392-0fb4d38879a7/add_router_interface HTTP/1.1" status: 200 len: 503 time: 4.3426645 May 20 04:28:38.246589 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:38.266376 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port a1e201be-75e9-4fbb-b3c4-47eeb5b0221c, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=a1e201be-75e9-4fbb-b3c4-47eeb5b0221c,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:28:38.271949 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:38.273994 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-ace8ac61-9ace-48af-93fb-1ea478ec7cb2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:38.348581 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=a1e201be-75e9-4fbb-b3c4-47eeb5b0221c,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:38.351942 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:38.778930 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-0bb6b2c3-c3f8-40b2-b8ee-f3a2c2ca4fbd tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-195167850&fields=id&tenant_id=0e3baa0ea24d42f3ab59e40845b350fd HTTP/1.1" status: 200 len: 267 time: 0.0550754 May 20 04:28:38.937036 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-50cf7964-229a-4b48-831e-3c5206a054d0 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:38.938463 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-50cf7964-229a-4b48-831e-3c5206a054d0 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/networks?id=14c537db-01aa-4360-a348-47f94c75e03b HTTP/1.1" status: 200 len: 810 time: 0.1528029 May 20 04:28:38.954171 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-99104fe3-fbdc-4671-9123-60d1cda65129 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/quotas/0e3baa0ea24d42f3ab59e40845b350fd HTTP/1.1" status: 200 len: 375 time: 0.0101981 May 20 04:28:39.049635 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-6fbaca6e-1866-42b2-b2f7-490b9544bfe1 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=0e3baa0ea24d42f3ab59e40845b350fd&fields=id HTTP/1.1" status: 200 len: 348 time: 0.0891469 May 20 04:28:39.430205 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-c236be83-0be5-494c-934c-2cd02805e0b9 service neutron] 10.222.0.39 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3635375 May 20 04:28:39.500283 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Sync routers for ids ['007d36bf-1482-45ff-a392-0fb4d38879a7'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:28:39.515026 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Routers {'007d36bf-1482-45ff-a392-0fb4d38879a7'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:28:39.524584 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:28:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['007d36bf-1482-45ff-a392-0fb4d38879a7'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:39.595964 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-95273ac0-635f-4629-8746-dca56f815253 service neutron] 10.222.0.39 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1557720 May 20 04:28:39.613513 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ae46ef7f-d45f-4108-a13f-2f457a871592 None None] New status for floating IP d2b40541-a315-4701-99bd-be17179c9b8b: DOWN {{(pid=91646) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 20 04:28:39.757254 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Set router 007d36bf-1482-45ff-a392-0fb4d38879a7 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:28:40.042409 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device a1e201be-75e9-4fbb-b3c4-47eeb5b0221c up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:28:40.243533 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port a1e201be-75e9-4fbb-b3c4-47eeb5b0221c triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:28:40.243919 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:40.352569 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ecc67d9c-5ba1-43b6-a199-bfef241cd27c req-882bfc94-c46d-4e74-ba09-f5355d470ff4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 210 time: 0.0697646 May 20 04:28:40.353780 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '007d36bf-1482-45ff-a392-0fb4d38879a7', 'name': 'tempest-TestMinimumBasicScenario-1656849615-router', 'tenant_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.119'}], 'enable_snat': True}, 'gw_port_id': 'a1e201be-75e9-4fbb-b3c4-47eeb5b0221c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:31Z', 'updated_at': '2021-05-20T04:28:36Z', 'revision_number': 4, 'project_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'gw_port': {'id': 'a1e201be-75e9-4fbb-b3c4-47eeb5b0221c', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6e:9b:ea', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '007d36bf-1482-45ff-a392-0fb4d38879a7', 'device_owner': 'network:router_gateway', 'standard_attr_id': 556, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.119', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:31Z', 'updated_at': '2021-05-20T04:28:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '8b38f061-73b2-48b5-bd6f-a15baf789252', 'name': '', 'network_id': 'c4d0b06b-6db7-4af5-9ff8-304cb32a79cd', 'tenant_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'mac_address': 'fa:16:3e:1a:b6:5c', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '007d36bf-1482-45ff-a392-0fb4d38879a7', 'device_owner': 'network:router_interface', 'standard_attr_id': 557, 'fixed_ips': [{'subnet_id': 'ce429852-561a-4890-b1d6-22338214dd01', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:35Z', 'updated_at': '2021-05-20T04:28:36Z', 'revision_number': 2, 'project_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'subnets': [{'id': 'ce429852-561a-4890-b1d6-22338214dd01', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:40.396639 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Checking router: 007d36bf-1482-45ff-a392-0fb4d38879a7 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:28:40.397237 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Updating router 007d36bf-1482-45ff-a392-0fb4d38879a7 port 8b38f061-73b2-48b5-bd6f-a15baf789252 binding host dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 20 04:28:40.479946 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:40.968842 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Notify callbacks [] for port, precommit_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:40.995661 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Transition to ACTIVE for port object 8b38f061-73b2-48b5-bd6f-a15baf789252 will not be triggered until provisioned by entity L2. {{(pid=91645) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:28:41.009084 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:41.015734 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:41.263165 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3aeed9b4-787d-4c22-8aed-9b60be95736d None None] Pushing event updated for resources: {'Port': ['ID=8b38f061-73b2-48b5-bd6f-a15baf789252,revision_number=3']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:41.272962 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-3aeed9b4-787d-4c22-8aed-9b60be95736d None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.257s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:41.368951 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Attempting to bind port 8b38f061-73b2-48b5-bd6f-a15baf789252 on host dv-cinder-782405-9-iscsi for vnic_type normal with profile {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:28:41.369548 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Attempting to bind port 8b38f061-73b2-48b5-bd6f-a15baf789252 by drivers openvswitch,hyperv on host dv-cinder-782405-9-iscsi at level 0 using segments [{'id': 'e3e8d115-0d77-4bf0-a504-52c3fb21936c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 838, 'network_id': 'c4d0b06b-6db7-4af5-9ff8-304cb32a79cd'}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:28:41.369981 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Attempting to bind port 8b38f061-73b2-48b5-bd6f-a15baf789252 on network c4d0b06b-6db7-4af5-9ff8-304cb32a79cd {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:28:41.377629 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Checking agent: {'id': 'c34f1378-a6a9-4e75-842f-5788796cfc67', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-782405-9-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 14, 15), 'started_at': datetime.datetime(2021, 5, 20, 4, 14, 45), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 28, 15), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-782405-9-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:28:41.378206 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Checking segment: {'id': 'e3e8d115-0d77-4bf0-a504-52c3fb21936c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 838, 'network_id': 'c4d0b06b-6db7-4af5-9ff8-304cb32a79cd'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91645) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:28:41.379076 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:41.379612 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Bound using segment: {'id': 'e3e8d115-0d77-4bf0-a504-52c3fb21936c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 838, 'network_id': 'c4d0b06b-6db7-4af5-9ff8-304cb32a79cd'} {{(pid=91645) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:28:41.380142 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.managers [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Bound port: 8b38f061-73b2-48b5-bd6f-a15baf789252, host: dv-cinder-782405-9-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'e3e8d115-0d77-4bf0-a504-52c3fb21936c', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 838, 'network_id': 'c4d0b06b-6db7-4af5-9ff8-304cb32a79cd'}}] {{(pid=91645) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:28:41.380576 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:41.538264 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] For port 8b38f061-73b2-48b5-bd6f-a15baf789252, host dv-cinder-782405-9-iscsi, cleared binding levels {{(pid=91645) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:28:41.548205 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] For port 8b38f061-73b2-48b5-bd6f-a15baf789252, host dv-cinder-782405-9-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=8b38f061-73b2-48b5-bd6f-a15baf789252,segment=NetworkSegment(e3e8d115-0d77-4bf0-a504-52c3fb21936c),segment_id=e3e8d115-0d77-4bf0-a504-52c3fb21936c)] {{(pid=91645) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:28:41.556664 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:41.563716 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:41.567850 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bc10e926-6ed7-4428-8485-90a66c2487db req-5d3300c2-9c13-410f-9878-788ad97c7cbd tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 210 time: 0.0574164 May 20 04:28:41.660034 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3aeed9b4-787d-4c22-8aed-9b60be95736d None None] Pushing event updated for resources: {'Port': ['ID=8b38f061-73b2-48b5-bd6f-a15baf789252,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:41.664601 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-3aeed9b4-787d-4c22-8aed-9b60be95736d None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.101s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:41.673564 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Routers {'007d36bf-1482-45ff-a392-0fb4d38879a7'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:28:41.674447 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:28:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['007d36bf-1482-45ff-a392-0fb4d38879a7'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:42.004073 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-99195520-a730-4249-aa0f-7cbe190fb481 req-984748c8-c08d-4695-bb15-6e29672dfe9a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:42.011760 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] Set router 007d36bf-1482-45ff-a392-0fb4d38879a7 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:28:42.014403 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-99195520-a730-4249-aa0f-7cbe190fb481 req-984748c8-c08d-4695-bb15-6e29672dfe9a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 946 time: 0.3788688 May 20 04:28:42.040204 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-7d77f15d-53f6-400c-97a6-5566367cbdcc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:42.042156 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-7d77f15d-53f6-400c-97a6-5566367cbdcc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.74 "GET /v2.0/networks?id=14c537db-01aa-4360-a348-47f94c75e03b HTTP/1.1" status: 200 len: 810 time: 0.4683716 May 20 04:28:42.142558 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-99195520-a730-4249-aa0f-7cbe190fb481 req-c3a0a0fb-a126-4531-8065-8853f12b027a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1273310 May 20 04:28:42.149455 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-1406dd00-6007-4808-82eb-b95b4897745b tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.74 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=0e3baa0ea24d42f3ab59e40845b350fd HTTP/1.1" status: 200 len: 379 time: 0.0950773 May 20 04:28:42.161693 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-c64b6be0-ec29-4200-b427-206c85899373 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Request body: {'port': {'device_id': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'admin_state_up': True, 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'security_groups': ['5a7943e2-185b-4112-9281-61ab71b1afeb']}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:28:42.319872 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-c64b6be0-ec29-4200-b427-206c85899373 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:28:42.333439 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-c64b6be0-ec29-4200-b427-206c85899373 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:28:42.346353 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-c64b6be0-ec29-4200-b427-206c85899373 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Made reservation on behalf of 0e3baa0ea24d42f3ab59e40845b350fd for: {'port': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:28:42.350496 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-c64b6be0-ec29-4200-b427-206c85899373 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:42.402202 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '007d36bf-1482-45ff-a392-0fb4d38879a7', 'name': 'tempest-TestMinimumBasicScenario-1656849615-router', 'tenant_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.119'}], 'enable_snat': True}, 'gw_port_id': 'a1e201be-75e9-4fbb-b3c4-47eeb5b0221c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:31Z', 'updated_at': '2021-05-20T04:28:36Z', 'revision_number': 4, 'project_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'gw_port': {'id': 'a1e201be-75e9-4fbb-b3c4-47eeb5b0221c', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6e:9b:ea', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '007d36bf-1482-45ff-a392-0fb4d38879a7', 'device_owner': 'network:router_gateway', 'standard_attr_id': 556, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.119', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:31Z', 'updated_at': '2021-05-20T04:28:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '8b38f061-73b2-48b5-bd6f-a15baf789252', 'name': '', 'network_id': 'c4d0b06b-6db7-4af5-9ff8-304cb32a79cd', 'tenant_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'mac_address': 'fa:16:3e:1a:b6:5c', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '007d36bf-1482-45ff-a392-0fb4d38879a7', 'device_owner': 'network:router_interface', 'standard_attr_id': 557, 'fixed_ips': [{'subnet_id': 'ce429852-561a-4890-b1d6-22338214dd01', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:35Z', 'updated_at': '2021-05-20T04:28:41Z', 'revision_number': 4, 'project_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'subnets': [{'id': 'ce429852-561a-4890-b1d6-22338214dd01', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:42.407871 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7b77ac17-4fb1-422d-abb1-e0121ac1d1ac None None] The sync data for ids ['007d36bf-1482-45ff-a392-0fb4d38879a7'] in dv-cinder-782405-9-iscsi is: [{'id': '007d36bf-1482-45ff-a392-0fb4d38879a7', 'name': 'tempest-TestMinimumBasicScenario-1656849615-router', 'tenant_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.119'}], 'enable_snat': True}, 'gw_port_id': 'a1e201be-75e9-4fbb-b3c4-47eeb5b0221c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:31Z', 'updated_at': '2021-05-20T04:28:36Z', 'revision_number': 4, 'project_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'gw_port': {'id': 'a1e201be-75e9-4fbb-b3c4-47eeb5b0221c', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6e:9b:ea', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '007d36bf-1482-45ff-a392-0fb4d38879a7', 'device_owner': 'network:router_gateway', 'standard_attr_id': 556, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.119', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:31Z', 'updated_at': '2021-05-20T04:28:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '8b38f061-73b2-48b5-bd6f-a15baf789252', 'name': '', 'network_id': 'c4d0b06b-6db7-4af5-9ff8-304cb32a79cd', 'tenant_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'mac_address': 'fa:16:3e:1a:b6:5c', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '007d36bf-1482-45ff-a392-0fb4d38879a7', 'device_owner': 'network:router_interface', 'standard_attr_id': 557, 'fixed_ips': [{'subnet_id': 'ce429852-561a-4890-b1d6-22338214dd01', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:35Z', 'updated_at': '2021-05-20T04:28:41Z', 'revision_number': 4, 'project_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'subnets': [{'id': 'ce429852-561a-4890-b1d6-22338214dd01', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:28:42.626897 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-01f79f77-89b5-49b4-8629-b71c72af6fe6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:42.648716 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-c64b6be0-ec29-4200-b427-206c85899373 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Allocated IP 172.20.0.5 (14c537db-01aa-4360-a348-47f94c75e03b/cdea475c-2929-4ae4-9271-87cc2fb9a378/ca396585-cdb5-4a50-a4c9-d5954043208e) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:28:43.044010 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1dd75d14-d50a-484d-a83f-7b8dac4723be req-e1d1f51e-49b0-447f-ac2d-c27bc887a64b tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 210 time: 0.0610247 May 20 04:28:43.076236 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-c64b6be0-ec29-4200-b427-206c85899373 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:43.104078 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.provisioning_blocks [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-c64b6be0-ec29-4200-b427-206c85899373 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Transition to ACTIVE for port object ca396585-cdb5-4a50-a4c9-d5954043208e will not be triggered until provisioned by entity DHCP. {{(pid=91643) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:28:43.145254 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-c64b6be0-ec29-4200-b427-206c85899373 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:43.151638 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:43.226636 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-c64b6be0-ec29-4200-b427-206c85899373 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Network 14c537db-01aa-4360-a348-47f94c75e03b is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:28:43.229834 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-c64b6be0-ec29-4200-b427-206c85899373 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:28:43.233878 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-c64b6be0-ec29-4200-b427-206c85899373 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:28:43.284963 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c64b6be0-ec29-4200-b427-206c85899373 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event updated for resources: {'Port': ['ID=ca396585-cdb5-4a50-a4c9-d5954043208e,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:43.296256 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-c64b6be0-ec29-4200-b427-206c85899373 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.144s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:43.360857 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-c64b6be0-ec29-4200-b427-206c85899373 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:43.361447 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-c64b6be0-ec29-4200-b427-206c85899373 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:43.505464 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-8427a830-de1c-4bc5-9088-75bed4d373a5 None None] Provisioning for port ca396585-cdb5-4a50-a4c9-d5954043208e completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:28:43.513490 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-8427a830-de1c-4bc5-9088-75bed4d373a5 None None] Provisioning complete for port ca396585-cdb5-4a50-a4c9-d5954043208e triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:28:43.513906 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-8427a830-de1c-4bc5-9088-75bed4d373a5 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:43.559257 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-c64b6be0-ec29-4200-b427-206c85899373 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.4034669 May 20 04:28:43.581197 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Request body: {'port': {'device_id': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:28:43.614208 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-8427a830-de1c-4bc5-9088-75bed4d373a5 None None] Port ca396585-cdb5-4a50-a4c9-d5954043208e cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:28:43.641131 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-398412ae-d006-47af-afc1-8fad178fb4a2 req-3d79c102-9189-4d3a-bcde-614788b14bee tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:43.642527 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-398412ae-d006-47af-afc1-8fad178fb4a2 req-3d79c102-9189-4d3a-bcde-614788b14bee tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Synchronizing usage tracker for tenant:9fafdf3a29024ac588e5b24719735677 on resource:port {{(pid=91644) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:28:43.659341 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-398412ae-d006-47af-afc1-8fad178fb4a2 req-3d79c102-9189-4d3a-bcde-614788b14bee tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Unset dirty status for tenant:9fafdf3a29024ac588e5b24719735677 on resource:port {{(pid=91644) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:28:43.662647 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-398412ae-d006-47af-afc1-8fad178fb4a2 req-3d79c102-9189-4d3a-bcde-614788b14bee tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.74 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 946 time: 0.1320553 May 20 04:28:43.716667 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-09d2ec68-ac5c-43a3-b002-abe494b1c498 req-ec09e4de-215c-40e7-96e8-56a23dabb6f0 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:43.718479 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-09d2ec68-ac5c-43a3-b002-abe494b1c498 req-ec09e4de-215c-40e7-96e8-56a23dabb6f0 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 946 time: 0.4074340 May 20 04:28:43.790139 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-09d2ec68-ac5c-43a3-b002-abe494b1c498 req-02c2c9b1-e820-41ea-9480-5b037ae9dd59 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=202eefda-65f8-4304-8243-021366a25be9&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0635598 May 20 04:28:43.790863 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:43.882911 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [req-398412ae-d006-47af-afc1-8fad178fb4a2 req-bc20fefc-08c6-4947-a736-adc08caa5b7b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Deleting port 535d5637-ae66-43cb-bd16-3543bd3b7a8b {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:28:43.883398 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-398412ae-d006-47af-afc1-8fad178fb4a2 req-bc20fefc-08c6-4947-a736-adc08caa5b7b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:44.010510 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 8b38f061-73b2-48b5-bd6f-a15baf789252 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:28:44.097913 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-398412ae-d006-47af-afc1-8fad178fb4a2 req-bc20fefc-08c6-4947-a736-adc08caa5b7b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] For port 535d5637-ae66-43cb-bd16-3543bd3b7a8b, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=535d5637-ae66-43cb-bd16-3543bd3b7a8b,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:28:44.101071 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-398412ae-d006-47af-afc1-8fad178fb4a2 req-bc20fefc-08c6-4947-a736-adc08caa5b7b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:44.123004 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning for port 8b38f061-73b2-48b5-bd6f-a15baf789252 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:28:44.141268 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 8b38f061-73b2-48b5-bd6f-a15baf789252 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:28:44.141660 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:44.354551 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id ca396585-cdb5-4a50-a4c9-d5954043208e {{(pid=91643) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:28:44.361583 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:44.364196 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Notify callbacks [] for port, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:44.386499 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.provisioning_blocks [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Transition to ACTIVE for port object ca396585-cdb5-4a50-a4c9-d5954043208e will not be triggered until provisioned by entity L2. {{(pid=91643) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:28:44.400377 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:44.406901 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-062bb502-043d-4a53-928e-b2bca65fe059 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:44.435378 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-198357cd-aaad-4397-a100-ced747a19175 None None] For port 8b38f061-73b2-48b5-bd6f-a15baf789252, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=8b38f061-73b2-48b5-bd6f-a15baf789252,segment=NetworkSegment(e3e8d115-0d77-4bf0-a504-52c3fb21936c),segment_id=e3e8d115-0d77-4bf0-a504-52c3fb21936c)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:28:44.442114 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:44.444194 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:44.485578 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Pushing event updated for resources: {'Port': ['ID=ca396585-cdb5-4a50-a4c9-d5954043208e,revision_number=2']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:44.489266 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.082s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:44.497458 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-250c3209-b0ea-4b9d-9faf-4a780dd5b192 req-d3c1e43c-5efe-48d3-9f5e-034fa1b2de42 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:44.498546 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-250c3209-b0ea-4b9d-9faf-4a780dd5b192 req-d3c1e43c-5efe-48d3-9f5e-034fa1b2de42 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Synchronizing usage tracker for tenant:0e3baa0ea24d42f3ab59e40845b350fd on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:28:44.555260 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Pushing event updated for resources: {'Port': ['ID=8b38f061-73b2-48b5-bd6f-a15baf789252,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:44.558602 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:44.608125 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-398412ae-d006-47af-afc1-8fad178fb4a2 req-bc20fefc-08c6-4947-a736-adc08caa5b7b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for floatingip, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:44.608549 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-398412ae-d006-47af-afc1-8fad178fb4a2 req-bc20fefc-08c6-4947-a736-adc08caa5b7b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-8003251'] for floatingip, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:44.608864 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.db.l3_db [req-398412ae-d006-47af-afc1-8fad178fb4a2 req-bc20fefc-08c6-4947-a736-adc08caa5b7b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Floating IP 2991a3a7-ad3f-4bb3-a8ca-e45809c0383d disassociated. External IP: 172.27.0.153, port: 535d5637-ae66-43cb-bd16-3543bd3b7a8b. May 20 04:28:44.609203 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [req-398412ae-d006-47af-afc1-8fad178fb4a2 req-bc20fefc-08c6-4947-a736-adc08caa5b7b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Calling delete_port for 535d5637-ae66-43cb-bd16-3543bd3b7a8b owned by compute:nova {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:28:44.666231 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-250c3209-b0ea-4b9d-9faf-4a780dd5b192 req-d3c1e43c-5efe-48d3-9f5e-034fa1b2de42 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Unset dirty status for tenant:0e3baa0ea24d42f3ab59e40845b350fd on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:28:44.673706 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-250c3209-b0ea-4b9d-9faf-4a780dd5b192 req-d3c1e43c-5efe-48d3-9f5e-034fa1b2de42 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 931 time: 0.4028692 May 20 04:28:44.755768 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-250c3209-b0ea-4b9d-9faf-4a780dd5b192 req-d0472f89-2fe8-4bda-83ce-875ca81f2a92 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0810313 May 20 04:28:44.761902 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-398412ae-d006-47af-afc1-8fad178fb4a2 req-bc20fefc-08c6-4947-a736-adc08caa5b7b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:44.769120 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-3aeb4226-67d6-4d52-919b-ef6cf2238a3c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:44.778346 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Attempting to bind port ca396585-cdb5-4a50-a4c9-d5954043208e on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:28:44.778953 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Attempting to bind port ca396585-cdb5-4a50-a4c9-d5954043208e by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'}] {{(pid=91643) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:28:44.780054 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Attempting to bind port ca396585-cdb5-4a50-a4c9-d5954043208e on network 14c537db-01aa-4360-a348-47f94c75e03b {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:28:44.788348 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Port ca396585-cdb5-4a50-a4c9-d5954043208e on network 14c537db-01aa-4360-a348-47f94c75e03b not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:28:44.788956 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Attempting to bind port ca396585-cdb5-4a50-a4c9-d5954043208e on network 14c537db-01aa-4360-a348-47f94c75e03b {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:28:44.795092 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 28, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:28:44.795530 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Checking segment: {'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91643) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:28:44.795908 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Bound using segment: {'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'} {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:28:44.796341 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Bound port: ca396585-cdb5-4a50-a4c9-d5954043208e, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'}}] {{(pid=91643) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:28:44.796684 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:44.818943 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bc20fefc-08c6-4947-a736-adc08caa5b7b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Pushing event deleted for resources: {'Port': ['ID=535d5637-ae66-43cb-bd16-3543bd3b7a8b,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:44.822066 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-bc20fefc-08c6-4947-a736-adc08caa5b7b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:44.915087 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] For port ca396585-cdb5-4a50-a4c9-d5954043208e, host hv-cinder-78240, cleared binding levels {{(pid=91643) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:28:44.918240 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] For port ca396585-cdb5-4a50-a4c9-d5954043208e, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=ca396585-cdb5-4a50-a4c9-d5954043208e,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91643) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:28:44.926140 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:44.989198 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-c64b6be0-ec29-4200-b427-206c85899373 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:45.000248 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:45.029839 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8d92ed92-fb69-4ec4-94a4-3eddd295460e req-78ce9a47-6532-4453-a8d2-f3b2f8b41b16 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 210 time: 0.0960598 May 20 04:28:45.083710 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:45.084227 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:45.086041 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] 10.222.0.74 "PUT /v2.0/ports/ca396585-cdb5-4a50-a4c9-d5954043208e HTTP/1.1" status: 200 len: 1093 time: 1.5143588 May 20 04:28:45.086694 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'tag': 'ca396585-cdb5-4a50-a4c9-d5954043208e'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:28:45.101742 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [req-398412ae-d006-47af-afc1-8fad178fb4a2 req-bc20fefc-08c6-4947-a736-adc08caa5b7b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91644) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:28:45.105552 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-dc716096-00d5-4216-9b5a-be2ffe1ec240 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:45.138636 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Pushing event updated for resources: {'Port': ['ID=ca396585-cdb5-4a50-a4c9-d5954043208e,revision_number=3']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:45.188688 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'tag': 'ca396585-cdb5-4a50-a4c9-d5954043208e', 'status': 'completed', 'code': 200} May 20 04:28:45.189531 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.201s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:45.262280 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-204b05a5-e383-4db7-adba-97296b981518 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:45.264160 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-204b05a5-e383-4db7-adba-97296b981518 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=0e3baa0ea24d42f3ab59e40845b350fd&device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 1096 time: 0.1225173 May 20 04:28:45.389997 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-37524a5a-5cbd-4af1-8c5a-48c631629598 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=ca396585-cdb5-4a50-a4c9-d5954043208e HTTP/1.1" status: 200 len: 217 time: 0.1168721 May 20 04:28:45.465861 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-09902ac5-eaf4-4e66-9382-d88218846481 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:45.467854 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-09902ac5-eaf4-4e66-9382-d88218846481 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=cdea475c-2929-4ae4-9271-87cc2fb9a378 HTTP/1.1" status: 200 len: 849 time: 0.0714517 May 20 04:28:45.549325 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-398412ae-d006-47af-afc1-8fad178fb4a2 req-bc20fefc-08c6-4947-a736-adc08caa5b7b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:28:45.560603 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-54cd0f98-e0e1-4035-80ab-fc0d9bf92d73 None None] Sync routers for ids ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:28:45.578645 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-54cd0f98-e0e1-4035-80ab-fc0d9bf92d73 None None] Routers {'5a3f69b9-fd58-499f-bec9-75215cd7f8fc'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:28:45.579745 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-54cd0f98-e0e1-4035-80ab-fc0d9bf92d73 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 9, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 9, "log_agent_heartbeats": false, "routers": 9},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:28:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:45.586102 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-80008800-9c72-49e3-8d9f-ccb3fa676ba9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:45.588931 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-80008800-9c72-49e3-8d9f-ccb3fa676ba9 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=14c537db-01aa-4360-a348-47f94c75e03b&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1130621 May 20 04:28:45.630829 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-398412ae-d006-47af-afc1-8fad178fb4a2 req-bc20fefc-08c6-4947-a736-adc08caa5b7b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] No DVR routers for this DVR port 535d5637-ae66-43cb-bd16-3543bd3b7a8b on host hv-cinder-78240 {{(pid=91644) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:28:45.635196 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-398412ae-d006-47af-afc1-8fad178fb4a2 req-bc20fefc-08c6-4947-a736-adc08caa5b7b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:45.638060 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'c2ca950f-78ff-4e44-9231-e8e19f4aee5d', 'name': 'network-vif-deleted', 'tag': '535d5637-ae66-43cb-bd16-3543bd3b7a8b'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:28:45.669302 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-398412ae-d006-47af-afc1-8fad178fb4a2 req-bc20fefc-08c6-4947-a736-adc08caa5b7b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.74 "DELETE /v2.0/ports/535d5637-ae66-43cb-bd16-3543bd3b7a8b HTTP/1.1" status: 204 len: 173 time: 1.9948912 May 20 04:28:45.727102 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'c2ca950f-78ff-4e44-9231-e8e19f4aee5d', 'name': 'network-vif-deleted', 'tag': '535d5637-ae66-43cb-bd16-3543bd3b7a8b', 'status': 'completed', 'code': 200} May 20 04:28:45.781407 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-7c29a055-310e-488a-9aa2-22e3b8424e84 service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1818724 May 20 04:28:45.955062 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-54cd0f98-e0e1-4035-80ab-fc0d9bf92d73 None None] Set router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:28:46.056260 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 8b38f061-73b2-48b5-bd6f-a15baf789252 up at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:28:46.129734 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4b466be8-9105-4a57-ab69-5a799fd67290 req-5a202233-1b3b-476d-933c-5c09f837d69d service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3426261 May 20 04:28:46.166999 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-32526fe7-4c8f-401c-85ed-10fb73e8953b req-a0521503-6958-401f-aa08-21af026f6767 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:46.168702 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-32526fe7-4c8f-401c-85ed-10fb73e8953b req-a0521503-6958-401f-aa08-21af026f6767 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 943 time: 0.1613925 May 20 04:28:46.363853 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-32526fe7-4c8f-401c-85ed-10fb73e8953b req-cf3b0f7b-8935-42d7-8802-f702d4b63392 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1808674 May 20 04:28:46.380680 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-eb114311-85be-4087-a03d-cad0ec63313a req-4feb01b0-46ef-4021-a55d-a9267ea53880 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=c2ca950f-78ff-4e44-9231-e8e19f4aee5d HTTP/1.1" status: 200 len: 210 time: 0.0823855 May 20 04:28:46.428331 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:46.429098 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bb462d0a-bc0f-4adc-be50-da2782b89eff req-6e5376c1-f46a-40ab-b025-03b044a88094 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:46.430923 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bb462d0a-bc0f-4adc-be50-da2782b89eff req-6e5376c1-f46a-40ab-b025-03b044a88094 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=0e3baa0ea24d42f3ab59e40845b350fd&device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 1096 time: 0.2074831 May 20 04:28:46.434918 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Provisioning complete for port 8b38f061-73b2-48b5-bd6f-a15baf789252 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:28:46.435385 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:46.510447 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-54cd0f98-e0e1-4035-80ab-fc0d9bf92d73 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port': {'id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:e4:9a:7c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:39Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'ac9495b7-5868-4104-85c9-3567f1350467', 'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'mac_address': 'fa:16:3e:12:5b:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_interface', 'standard_attr_id': 82, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:45Z', 'updated_at': '2021-05-20T04:21:55Z', 'revision_number': 5, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'subnets': [{'id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:46.541820 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-54cd0f98-e0e1-4035-80ab-fc0d9bf92d73 None None] Checking router: 5a3f69b9-fd58-499f-bec9-75215cd7f8fc for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:28:46.567480 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-54cd0f98-e0e1-4035-80ab-fc0d9bf92d73 None None] Routers {'5a3f69b9-fd58-499f-bec9-75215cd7f8fc'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:28:46.568156 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-54cd0f98-e0e1-4035-80ab-fc0d9bf92d73 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 9, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 9, "log_agent_heartbeats": false, "routers": 9},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:28:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:46.581967 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bb462d0a-bc0f-4adc-be50-da2782b89eff req-260c2679-f09c-4b5f-b961-a5d0faee4452 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:46.583721 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bb462d0a-bc0f-4adc-be50-da2782b89eff req-260c2679-f09c-4b5f-b961-a5d0faee4452 service neutron] 10.222.0.74 "GET /v2.0/networks?id=14c537db-01aa-4360-a348-47f94c75e03b HTTP/1.1" status: 200 len: 911 time: 0.1446471 May 20 04:28:46.681620 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bb462d0a-bc0f-4adc-be50-da2782b89eff req-2fe0ca77-8fd2-4c25-8bcf-6bfe40026e04 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=ca396585-cdb5-4a50-a4c9-d5954043208e HTTP/1.1" status: 200 len: 217 time: 0.0900042 May 20 04:28:46.760016 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-54cd0f98-e0e1-4035-80ab-fc0d9bf92d73 None None] Set router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:28:46.763841 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bb462d0a-bc0f-4adc-be50-da2782b89eff req-6aab26c1-842e-49bc-b38a-e69f3f312995 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:46.765832 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bb462d0a-bc0f-4adc-be50-da2782b89eff req-6aab26c1-842e-49bc-b38a-e69f3f312995 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=cdea475c-2929-4ae4-9271-87cc2fb9a378 HTTP/1.1" status: 200 len: 849 time: 0.0766370 May 20 04:28:46.851822 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bb462d0a-bc0f-4adc-be50-da2782b89eff req-4255a1c9-ac54-4945-a702-e422db54fe3d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:46.854232 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bb462d0a-bc0f-4adc-be50-da2782b89eff req-4255a1c9-ac54-4945-a702-e422db54fe3d service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=14c537db-01aa-4360-a348-47f94c75e03b&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0812919 May 20 04:28:47.026614 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bb462d0a-bc0f-4adc-be50-da2782b89eff req-9bbd269e-586b-4e48-bbc0-617ff21a4821 service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1644523 May 20 04:28:47.104238 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-54cd0f98-e0e1-4035-80ab-fc0d9bf92d73 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port': {'id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:e4:9a:7c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:39Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'ac9495b7-5868-4104-85c9-3567f1350467', 'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'mac_address': 'fa:16:3e:12:5b:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_interface', 'standard_attr_id': 82, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:45Z', 'updated_at': '2021-05-20T04:21:55Z', 'revision_number': 5, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'subnets': [{'id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:28:47.108719 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-54cd0f98-e0e1-4035-80ab-fc0d9bf92d73 None None] The sync data for ids ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'] in dv-cinder-782405-9-iscsi is: [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port': {'id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:e4:9a:7c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:39Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'ac9495b7-5868-4104-85c9-3567f1350467', 'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'mac_address': 'fa:16:3e:12:5b:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_interface', 'standard_attr_id': 82, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:45Z', 'updated_at': '2021-05-20T04:21:55Z', 'revision_number': 5, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'subnets': [{'id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:28:47.191365 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bb462d0a-bc0f-4adc-be50-da2782b89eff req-4712eaf1-7400-4c40-bdb5-1b69f8cb6860 service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1583200 May 20 04:28:47.392638 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-7f0c305f-24cd-4db7-a38d-67444e52db02 None None] New status for floating IP 2991a3a7-ad3f-4bb3-a8ca-e45809c0383d: DOWN {{(pid=91646) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 20 04:28:47.594322 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 41466) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:28:47.765296 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-727d0d19-c9ea-4a3d-973d-fd34da44b6c8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:47.790698 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e1bd0b0-e707-409b-b326-01805d887152 req-622446d9-84fb-4726-9452-1c0ea9518bad tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:47.792271 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8e1bd0b0-e707-409b-b326-01805d887152 req-622446d9-84fb-4726-9452-1c0ea9518bad tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 943 time: 0.1733994 May 20 04:28:47.828393 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-727d0d19-c9ea-4a3d-973d-fd34da44b6c8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:47.836679 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 535d5637-ae66-43cb-bd16-3543bd3b7a8b no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:28:47.877714 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: 535d5637-ae66-43cb-bd16-3543bd3b7a8b {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:28:47.878063 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 535d5637-ae66-43cb-bd16-3543bd3b7a8b not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:28:47.886107 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8e1bd0b0-e707-409b-b326-01805d887152 req-30ebb09e-9b40-4d10-aa11-2df69e573283 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0826561 May 20 04:28:47.887529 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:47.943786 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-727d0d19-c9ea-4a3d-973d-fd34da44b6c8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:47.948039 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-32f347db-d0a9-4cee-adbd-ade2eceb514b tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:47.966148 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-727d0d19-c9ea-4a3d-973d-fd34da44b6c8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=202eefda-65f8-4304-8243-021366a25be9,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:47.974867 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-727d0d19-c9ea-4a3d-973d-fd34da44b6c8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:47.976966 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-727d0d19-c9ea-4a3d-973d-fd34da44b6c8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:47.982035 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-727d0d19-c9ea-4a3d-973d-fd34da44b6c8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "DELETE /v2.0/security-groups/202eefda-65f8-4304-8243-021366a25be9 HTTP/1.1" status: 204 len: 168 time: 0.3827088 May 20 04:28:47.982035 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 41470) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:28:47.998888 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.api.v2.resource [None req-22bf32ce-bf0e-4e2f-be7c-59a923496810 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] show failed (client error): The resource could not be found. May 20 04:28:48.003255 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-22bf32ce-bf0e-4e2f-be7c-59a923496810 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups/202eefda-65f8-4304-8243-021366a25be9 HTTP/1.1" status: 404 len: 347 time: 0.0178931 May 20 04:28:48.003255 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 41474) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:28:48.237403 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-84078db7-6f3a-45c8-ad1e-bc5b2c48a8d9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Deleting port 645c120d-b75d-4b37-a533-df616457b087 {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:28:48.237847 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-84078db7-6f3a-45c8-ad1e-bc5b2c48a8d9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:48.367788 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-84078db7-6f3a-45c8-ad1e-bc5b2c48a8d9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:48.368116 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-84078db7-6f3a-45c8-ad1e-bc5b2c48a8d9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:48.590068 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-84078db7-6f3a-45c8-ad1e-bc5b2c48a8d9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Calling delete_port for 645c120d-b75d-4b37-a533-df616457b087 owned by network:floatingip {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:28:48.712831 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-84078db7-6f3a-45c8-ad1e-bc5b2c48a8d9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:48.717503 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:48.765907 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-84078db7-6f3a-45c8-ad1e-bc5b2c48a8d9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Pushing event deleted for resources: {'Port': ['ID=645c120d-b75d-4b37-a533-df616457b087,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:48.768493 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-84078db7-6f3a-45c8-ad1e-bc5b2c48a8d9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:48.864486 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-84078db7-6f3a-45c8-ad1e-bc5b2c48a8d9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for floatingip, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:48.878911 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-84078db7-6f3a-45c8-ad1e-bc5b2c48a8d9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:48.880081 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-84078db7-6f3a-45c8-ad1e-bc5b2c48a8d9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "DELETE /v2.0/floatingips/2991a3a7-ad3f-4bb3-a8ca-e45809c0383d HTTP/1.1" status: 204 len: 168 time: 0.8762035 May 20 04:28:49.101112 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1ac06e19-6ba6-4669-98ee-e0a9c752febf req-4e4a2f5f-ac20-4a49-a1b9-7b9c5c37a809 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:49.102611 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1ac06e19-6ba6-4669-98ee-e0a9c752febf req-4e4a2f5f-ac20-4a49-a1b9-7b9c5c37a809 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 943 time: 0.1082096 May 20 04:28:49.156511 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1ac06e19-6ba6-4669-98ee-e0a9c752febf req-4e1d889a-2e44-49c2-99f0-c2ad32dda72b tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0475984 May 20 04:28:49.258361 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 41482) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:28:49.264715 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-f75ec2a5-15b3-4fa5-aaf0-69ad1b54c817 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Request body: {'security_group': {'name': 'tempest-securitygroup--698863212', 'description': 'tempest-description--989485486'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:28:49.273065 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-f75ec2a5-15b3-4fa5-aaf0-69ad1b54c817 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:28:49.278166 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-f75ec2a5-15b3-4fa5-aaf0-69ad1b54c817 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Usage tracker for resource:security_group and tenant:9fafdf3a29024ac588e5b24719735677 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:28:49.281209 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-f75ec2a5-15b3-4fa5-aaf0-69ad1b54c817 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:28:49.288905 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-f75ec2a5-15b3-4fa5-aaf0-69ad1b54c817 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:28:49.304361 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f75ec2a5-15b3-4fa5-aaf0-69ad1b54c817 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:49.329358 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-f75ec2a5-15b3-4fa5-aaf0-69ad1b54c817 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:28:49.338190 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-f75ec2a5-15b3-4fa5-aaf0-69ad1b54c817 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Usage tracker for resource:security_group_rule and tenant:9fafdf3a29024ac588e5b24719735677 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:28:49.341338 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-f75ec2a5-15b3-4fa5-aaf0-69ad1b54c817 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:28:49.348040 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-f75ec2a5-15b3-4fa5-aaf0-69ad1b54c817 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:28:49.431669 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f75ec2a5-15b3-4fa5-aaf0-69ad1b54c817 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:49.437154 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f75ec2a5-15b3-4fa5-aaf0-69ad1b54c817 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:49.442875 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-8d49c95b-ba15-43b5-bb02-6cc4515a16fe admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:49.485836 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f75ec2a5-15b3-4fa5-aaf0-69ad1b54c817 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Pushing event updated for resources: {'SecurityGroup': ['ID=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:49.489415 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-f75ec2a5-15b3-4fa5-aaf0-69ad1b54c817 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:49.490790 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f75ec2a5-15b3-4fa5-aaf0-69ad1b54c817 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:49.491580 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-f75ec2a5-15b3-4fa5-aaf0-69ad1b54c817 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1778 time: 0.2319281 May 20 04:28:49.494451 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 41486) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:28:49.499893 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-d1d128bd-78e4-43c0-937a-8aa0200c5bc2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Request body: {'security_group_rule': {'security_group_id': '7a1d2a6b-5b70-4cb9-a737-fa50141ab0be', 'protocol': 'tcp', 'ethertype': 'IPv4', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:28:49.506297 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-d1d128bd-78e4-43c0-937a-8aa0200c5bc2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:28:49.510164 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-d1d128bd-78e4-43c0-937a-8aa0200c5bc2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Usage tracker for resource:security_group_rule and tenant:9fafdf3a29024ac588e5b24719735677 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:28:49.514534 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-d1d128bd-78e4-43c0-937a-8aa0200c5bc2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:28:49.519328 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-d1d128bd-78e4-43c0-937a-8aa0200c5bc2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:28:49.539785 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d1d128bd-78e4-43c0-937a-8aa0200c5bc2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:49.605354 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d1d128bd-78e4-43c0-937a-8aa0200c5bc2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:49.612461 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d1d128bd-78e4-43c0-937a-8aa0200c5bc2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:49.614733 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-91a6ea50-f0ab-45f2-ae0c-8fdb223178c7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:49.637525 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d1d128bd-78e4-43c0-937a-8aa0200c5bc2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=b1394d34-d858-4a79-8fee-76786c17e7ec,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:49.643055 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-d1d128bd-78e4-43c0-937a-8aa0200c5bc2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:49.648585 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d1d128bd-78e4-43c0-937a-8aa0200c5bc2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:49.653827 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-d1d128bd-78e4-43c0-937a-8aa0200c5bc2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.1544821 May 20 04:28:49.654070 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 41490) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:28:49.659726 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-3e1c4efc-b6e6-45f9-a4f8-f7bbcc5f1c34 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Request body: {'security_group_rule': {'security_group_id': '7a1d2a6b-5b70-4cb9-a737-fa50141ab0be', 'protocol': 'icmp', 'ethertype': 'IPv4', 'direction': 'ingress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:28:49.666076 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-3e1c4efc-b6e6-45f9-a4f8-f7bbcc5f1c34 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:28:49.672418 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-3e1c4efc-b6e6-45f9-a4f8-f7bbcc5f1c34 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Usage tracker for resource:security_group_rule and tenant:9fafdf3a29024ac588e5b24719735677 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:28:49.675074 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-3e1c4efc-b6e6-45f9-a4f8-f7bbcc5f1c34 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:28:49.683575 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-3e1c4efc-b6e6-45f9-a4f8-f7bbcc5f1c34 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:28:49.702962 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3e1c4efc-b6e6-45f9-a4f8-f7bbcc5f1c34 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:49.776754 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3e1c4efc-b6e6-45f9-a4f8-f7bbcc5f1c34 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:49.785699 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3e1c4efc-b6e6-45f9-a4f8-f7bbcc5f1c34 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:49.788171 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-b07a5335-eff0-4f6d-a7e2-918d5e7ad195 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:49.819484 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3e1c4efc-b6e6-45f9-a4f8-f7bbcc5f1c34 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=8abc352a-9841-45eb-8868-d7739b15ecd2,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:49.822794 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-3e1c4efc-b6e6-45f9-a4f8-f7bbcc5f1c34 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.035s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:49.827296 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3e1c4efc-b6e6-45f9-a4f8-f7bbcc5f1c34 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:49.833317 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-3e1c4efc-b6e6-45f9-a4f8-f7bbcc5f1c34 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.1733620 May 20 04:28:49.833556 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 41494) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:28:49.837582 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Request body: {'floatingip': {'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:28:49.844181 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:28:49.850248 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Usage tracker for resource:floatingip and tenant:9fafdf3a29024ac588e5b24719735677 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:28:49.853338 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:28:49.860670 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:28:49.886308 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for floatingip, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:49.998331 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:28:50.032500 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:50.193919 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Allocated IP 172.27.0.70 (712ce8c4-8feb-4719-bf13-e06992125cd9/e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5/c8308b42-b870-4612-be13-022cf48bcb41) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:28:50.375807 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a0d71d7d-b1df-4b69-aac9-45ebb855544e req-ce1def68-2d01-466d-9995-2fd5412e0446 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:50.378067 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a0d71d7d-b1df-4b69-aac9-45ebb855544e req-ce1def68-2d01-466d-9995-2fd5412e0446 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 943 time: 0.1010556 May 20 04:28:50.383419 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:50.417619 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:50.422979 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:50.444065 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a0d71d7d-b1df-4b69-aac9-45ebb855544e req-c5e69e1f-2152-413e-a69c-0292d96da1a6 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0593817 May 20 04:28:50.486431 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:28:50.489685 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:28:50.493639 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:28:50.528190 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Pushing event updated for resources: {'Port': ['ID=c8308b42-b870-4612-be13-022cf48bcb41,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:50.532650 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.110s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:50.577725 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for floatingip, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:50.643680 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:50.851718 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:50.856795 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:50.861340 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:50.964653 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Pushing event updated for resources: {'Port': ['ID=c8308b42-b870-4612-be13-022cf48bcb41,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:50.968987 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.108s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:50.976476 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-8003251'] for floatingip, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:50.999445 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:51.008826 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 692 time: 1.1678543 May 20 04:28:51.295677 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-fca4ab59-6b30-4daa-843c-b59f32692396 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Synchronizing usage tracker for tenant:9fafdf3a29024ac588e5b24719735677 on resource:security_group {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:28:51.312446 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-91e2f431-05b2-4d4f-8072-0ca3629b666a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:51.313112 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-91e2f431-05b2-4d4f-8072-0ca3629b666a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Synchronizing usage tracker for tenant:13d0e12c764f483ea457246b10c4cf7e on resource:network {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:28:51.333082 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-fca4ab59-6b30-4daa-843c-b59f32692396 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Unset dirty status for tenant:9fafdf3a29024ac588e5b24719735677 on resource:security_group {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:28:51.334097 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-fca4ab59-6b30-4daa-843c-b59f32692396 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?name=tempest-securitygroup--698863212&fields=id&tenant_id=9fafdf3a29024ac588e5b24719735677 HTTP/1.1" status: 200 len: 267 time: 0.1091099 May 20 04:28:51.338588 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-91e2f431-05b2-4d4f-8072-0ca3629b666a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Unset dirty status for tenant:13d0e12c764f483ea457246b10c4cf7e on resource:network {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:28:51.339662 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-91e2f431-05b2-4d4f-8072-0ca3629b666a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/networks?id=9798d104-d4de-413f-b41d-fb9d7e8a818e HTTP/1.1" status: 200 len: 813 time: 0.2015972 May 20 04:28:51.437215 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-ad47796c-7475-4faf-8168-715b4b134fe6 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/quotas/13d0e12c764f483ea457246b10c4cf7e HTTP/1.1" status: 200 len: 375 time: 0.0668662 May 20 04:28:51.545255 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-0a8361e9-c518-447b-8bd3-469afbdd3782 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:51.546676 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-0a8361e9-c518-447b-8bd3-469afbdd3782 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/networks?id=64bcea8a-6d35-406b-a823-fa1185008174 HTTP/1.1" status: 200 len: 817 time: 0.2029796 May 20 04:28:51.565636 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-82d235ad-8678-4299-a233-1d4c723a0b80 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/quotas/9fafdf3a29024ac588e5b24719735677 HTTP/1.1" status: 200 len: 375 time: 0.0129812 May 20 04:28:51.576097 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-66875783-14c1-4c4b-94c9-20a8e84892e0 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Synchronizing usage tracker for tenant:13d0e12c764f483ea457246b10c4cf7e on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:28:51.637374 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-66875783-14c1-4c4b-94c9-20a8e84892e0 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Unset dirty status for tenant:13d0e12c764f483ea457246b10c4cf7e on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:28:51.638547 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-66875783-14c1-4c4b-94c9-20a8e84892e0 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=13d0e12c764f483ea457246b10c4cf7e&fields=id HTTP/1.1" status: 200 len: 302 time: 0.1958752 May 20 04:28:51.745238 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-21fd5ab3-41f3-4b4e-876e-a38838e5dbb6 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Synchronizing usage tracker for tenant:9fafdf3a29024ac588e5b24719735677 on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:28:51.784239 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-21fd5ab3-41f3-4b4e-876e-a38838e5dbb6 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Unset dirty status for tenant:9fafdf3a29024ac588e5b24719735677 on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:28:51.785523 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-21fd5ab3-41f3-4b4e-876e-a38838e5dbb6 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=9fafdf3a29024ac588e5b24719735677&fields=id HTTP/1.1" status: 200 len: 348 time: 0.2147434 May 20 04:28:51.823957 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-2f4110cc-d13f-4b2c-a1fa-a3a05baf6e5b service neutron] 10.222.0.39 "GET /v2.0/networks/9798d104-d4de-413f-b41d-fb9d7e8a818e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1730235 May 20 04:28:52.117579 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-9fa8dab8-6425-48ed-8e18-9a1b8f16831b service neutron] 10.222.0.39 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3184423 May 20 04:28:52.145865 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-666a128e-738b-4805-8a06-340e2ff89906 service neutron] 10.222.0.39 "GET /v2.0/networks/9798d104-d4de-413f-b41d-fb9d7e8a818e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3112326 May 20 04:28:52.257924 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-a6ec3dfe-9dc1-4bbd-83f3-328e24814d60 service neutron] 10.222.0.39 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1338093 May 20 04:28:52.463300 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4b9ef5f1-c7c5-4882-84b4-2f95163e6d64 req-b4d670a7-61aa-4093-ab31-242ae5314d02 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 210 time: 0.0603461 May 20 04:28:52.499658 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f6c7347c-1550-46cf-80ea-f71ae86edf8b req-fa01d6bb-14a2-405c-a79e-73f2b4a8ce8a tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:52.501098 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f6c7347c-1550-46cf-80ea-f71ae86edf8b req-fa01d6bb-14a2-405c-a79e-73f2b4a8ce8a tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 943 time: 0.1648254 May 20 04:28:52.598327 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-44ac81f7-ec55-4010-a95e-3b101663f4f6 req-e5090c5c-67c0-4ffd-8cc5-9a53832fd1ab tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 210 time: 0.0849082 May 20 04:28:52.635636 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f6c7347c-1550-46cf-80ea-f71ae86edf8b req-3efb36b1-0472-4da0-9561-0e1903505748 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1233771 May 20 04:28:53.550492 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-775071c6-864b-4ffe-9300-15b1a9bc9009 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:53.551957 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-775071c6-864b-4ffe-9300-15b1a9bc9009 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.74 "GET /v2.0/networks?id=9798d104-d4de-413f-b41d-fb9d7e8a818e HTTP/1.1" status: 200 len: 813 time: 0.1648180 May 20 04:28:53.564922 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-29f52356-0fe3-4c4f-a5f6-0002988a644a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Request body: {'port': {'device_id': 'dfc47d7d-eb29-4bb6-a680-660af78ffcf3', 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e', 'admin_state_up': True, 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:28:53.650056 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-737c3cf2-fd3d-4878-ae60-cd49b2c2ea5d req-aba7dff5-6e8f-4d2b-99a9-23bb260e8ede tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 210 time: 0.0659668 May 20 04:28:53.821518 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-928cb059-54e6-4c1d-836c-c1750c405d38 req-9c3c2122-f0ff-454a-a718-95cf49721c0c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 210 time: 0.1029775 May 20 04:28:53.863808 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-29f52356-0fe3-4c4f-a5f6-0002988a644a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:28:53.880719 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-29f52356-0fe3-4c4f-a5f6-0002988a644a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:28:53.899063 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-53e33169-d35f-45fa-b884-c215ebb76e87 req-78bb42f9-e51b-4523-bc07-ccd50c218de8 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:53.901241 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-53e33169-d35f-45fa-b884-c215ebb76e87 req-78bb42f9-e51b-4523-bc07-ccd50c218de8 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 943 time: 0.1349928 May 20 04:28:53.906941 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-29f52356-0fe3-4c4f-a5f6-0002988a644a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Made reservation on behalf of 13d0e12c764f483ea457246b10c4cf7e for: {'port': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:28:53.911378 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-29f52356-0fe3-4c4f-a5f6-0002988a644a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:54.067044 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-946b5412-3547-4f57-a70a-9dba65f59fae tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:54.067817 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-946b5412-3547-4f57-a70a-9dba65f59fae tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Synchronizing usage tracker for tenant:9fafdf3a29024ac588e5b24719735677 on resource:network {{(pid=91644) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:28:54.089138 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-946b5412-3547-4f57-a70a-9dba65f59fae tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Unset dirty status for tenant:9fafdf3a29024ac588e5b24719735677 on resource:network {{(pid=91644) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:28:54.090738 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-946b5412-3547-4f57-a70a-9dba65f59fae tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.74 "GET /v2.0/networks?id=64bcea8a-6d35-406b-a823-fa1185008174 HTTP/1.1" status: 200 len: 817 time: 0.2162206 May 20 04:28:54.205481 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-53e33169-d35f-45fa-b884-c215ebb76e87 req-57286dcb-0e83-4330-a547-557b93b6b074 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.2761104 May 20 04:28:54.207403 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-e82062ad-4128-46b9-a2f6-a793f4159cd4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Synchronizing usage tracker for tenant:9fafdf3a29024ac588e5b24719735677 on resource:security_group {{(pid=91644) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:28:54.236274 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-e82062ad-4128-46b9-a2f6-a793f4159cd4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Unset dirty status for tenant:9fafdf3a29024ac588e5b24719735677 on resource:security_group {{(pid=91644) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:28:54.241401 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-e82062ad-4128-46b9-a2f6-a793f4159cd4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.74 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=9fafdf3a29024ac588e5b24719735677 HTTP/1.1" status: 200 len: 379 time: 0.1272607 May 20 04:28:54.333068 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-29f52356-0fe3-4c4f-a5f6-0002988a644a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Allocated IP 172.20.0.12 (9798d104-d4de-413f-b41d-fb9d7e8a818e/c5423164-3d24-4562-871f-ab03914420dc/5f77cfe6-85e2-4a40-9a27-6ed37313ae13) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:28:54.624999 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-29f52356-0fe3-4c4f-a5f6-0002988a644a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:54.645513 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.provisioning_blocks [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-29f52356-0fe3-4c4f-a5f6-0002988a644a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Transition to ACTIVE for port object 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 will not be triggered until provisioned by entity DHCP. {{(pid=91643) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:28:54.672224 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e6381604-4331-4725-8d83-840a43573682 req-43259f4d-eb8b-4880-9381-1edef32a77c6 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:54.673704 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e6381604-4331-4725-8d83-840a43573682 req-43259f4d-eb8b-4880-9381-1edef32a77c6 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.1390276 May 20 04:28:54.729348 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-29f52356-0fe3-4c4f-a5f6-0002988a644a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:54.736162 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:54.776103 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e6381604-4331-4725-8d83-840a43573682 req-bf1de7de-3a59-405d-af1a-de608a8f1911 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0917516 May 20 04:28:54.855035 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-29f52356-0fe3-4c4f-a5f6-0002988a644a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Network 9798d104-d4de-413f-b41d-fb9d7e8a818e is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:28:54.864415 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-29f52356-0fe3-4c4f-a5f6-0002988a644a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:28:54.869335 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-29f52356-0fe3-4c4f-a5f6-0002988a644a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Pushing event updated for resources: {'Port': ['ID=5f77cfe6-85e2-4a40-9a27-6ed37313ae13,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:54.873419 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-29f52356-0fe3-4c4f-a5f6-0002988a644a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:28:54.899789 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-29f52356-0fe3-4c4f-a5f6-0002988a644a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.163s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:54.960291 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-29f52356-0fe3-4c4f-a5f6-0002988a644a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:54.960987 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-29f52356-0fe3-4c4f-a5f6-0002988a644a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:55.146250 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7c8b5f4f-96ae-404b-b646-af5a7f852106 req-55301657-d0db-49b9-a517-f9e37750a658 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:55.147713 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7c8b5f4f-96ae-404b-b646-af5a7f852106 req-55301657-d0db-49b9-a517-f9e37750a658 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 932 time: 0.3642547 May 20 04:28:55.179053 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-11d2b915-bce5-4282-a93c-d0cec57c30ff req-53be272d-83bb-4b56-96ac-b2d42a4df253 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 210 time: 0.2126663 May 20 04:28:55.191480 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-29f52356-0fe3-4c4f-a5f6-0002988a644a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 1.6304312 May 20 04:28:55.212632 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-7c8b5f4f-96ae-404b-b646-af5a7f852106 req-0b3ac195-d261-4a24-89dc-45beb646104b tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Synchronizing usage tracker for tenant:13d0e12c764f483ea457246b10c4cf7e on resource:security_group {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:28:55.232257 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-7c8b5f4f-96ae-404b-b646-af5a7f852106 req-0b3ac195-d261-4a24-89dc-45beb646104b tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Unset dirty status for tenant:13d0e12c764f483ea457246b10c4cf7e on resource:security_group {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:28:55.234030 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7c8b5f4f-96ae-404b-b646-af5a7f852106 req-0b3ac195-d261-4a24-89dc-45beb646104b tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0784333 May 20 04:28:55.239962 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-369eb9c6-ec03-4c59-bc90-152d39f59a33 None None] Provisioning for port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:28:55.248344 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-369eb9c6-ec03-4c59-bc90-152d39f59a33 None None] Provisioning complete for port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:28:55.248720 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-369eb9c6-ec03-4c59-bc90-152d39f59a33 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:55.324942 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-369eb9c6-ec03-4c59-bc90-152d39f59a33 None None] Port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:28:55.423554 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bc5d3c20-ecb4-4264-a1d1-209557538bad req-a680939d-bc15-4341-a6e4-f92a1c95cc11 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:55.425127 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bc5d3c20-ecb4-4264-a1d1-209557538bad req-a680939d-bc15-4341-a6e4-f92a1c95cc11 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 943 time: 0.0919015 May 20 04:28:55.472695 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bc5d3c20-ecb4-4264-a1d1-209557538bad req-f5b0ee13-675d-489e-aa06-1402f3c7d937 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0414619 May 20 04:28:55.881376 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device ca396585-cdb5-4a50-a4c9-d5954043208e details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:28:55.992467 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-880e66d5-0993-469f-8665-5ddd5b7eae91 req-3300b9ed-6cbc-419d-85fd-56368d40b28a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:55.994055 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-880e66d5-0993-469f-8665-5ddd5b7eae91 req-3300b9ed-6cbc-419d-85fd-56368d40b28a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0797813 May 20 04:28:56.019622 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port ca396585-cdb5-4a50-a4c9-d5954043208e, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=ca396585-cdb5-4a50-a4c9-d5954043208e,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:28:56.033296 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': 'ca396585-cdb5-4a50-a4c9-d5954043208e', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'port_id': 'ca396585-cdb5-4a50-a4c9-d5954043208e', 'mac_address': 'fa:16:3e:37:08:e4', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 841, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.5'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:28:56.038638 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-880e66d5-0993-469f-8665-5ddd5b7eae91 req-53865e1d-9e86-45d2-84fd-98a8874b6586 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0386422 May 20 04:28:56.098960 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:56.115929 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id ca396585-cdb5-4a50-a4c9-d5954043208e {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:28:56.131424 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port ca396585-cdb5-4a50-a4c9-d5954043208e, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=ca396585-cdb5-4a50-a4c9-d5954043208e,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:28:56.135969 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:56.138432 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-cf924b77-a1a6-4496-a46e-f71487656834 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:56.210705 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=ca396585-cdb5-4a50-a4c9-d5954043208e,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:56.217924 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.079s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:56.342207 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2d806873-60f7-4634-b1c7-aa4e865c787a req-fe51e03f-350a-400b-94a8-3f0f9bbe7e51 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 210 time: 0.0448301 May 20 04:28:56.441866 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-031a76ff-bb5d-44ff-9fa0-2d42f79e5523 req-b7c234b8-145f-4bf2-af41-bb2430e319d6 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:56.442554 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-031a76ff-bb5d-44ff-9fa0-2d42f79e5523 req-b7c234b8-145f-4bf2-af41-bb2430e319d6 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Synchronizing usage tracker for tenant:13d0e12c764f483ea457246b10c4cf7e on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:28:56.458963 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-031a76ff-bb5d-44ff-9fa0-2d42f79e5523 req-b7c234b8-145f-4bf2-af41-bb2430e319d6 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Unset dirty status for tenant:13d0e12c764f483ea457246b10c4cf7e on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:28:56.460279 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-031a76ff-bb5d-44ff-9fa0-2d42f79e5523 req-b7c234b8-145f-4bf2-af41-bb2430e319d6 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 932 time: 0.0988703 May 20 04:28:56.507069 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-031a76ff-bb5d-44ff-9fa0-2d42f79e5523 req-79ad903e-2ce9-4ca9-95db-969641a94bca tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0409856 May 20 04:28:56.522860 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['ca396585-cdb5-4a50-a4c9-d5954043208e'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:28:56.674423 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1bf52b65-e42c-4440-a255-494fe5793ecc req-372e7e8a-ede0-4c1d-9a3e-f0901f207a32 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:56.682040 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1bf52b65-e42c-4440-a255-494fe5793ecc req-372e7e8a-ede0-4c1d-9a3e-f0901f207a32 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 944 time: 0.0909290 May 20 04:28:56.724790 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1bf52b65-e42c-4440-a255-494fe5793ecc req-4dd9cd7c-77b4-4630-a964-f1db9ac7dcd0 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0418720 May 20 04:28:57.232140 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f681fd08-a9b7-418a-8d11-87ea03db4101 req-232e276a-f5c7-4c7d-b605-27e7a851600d tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:57.233598 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f681fd08-a9b7-418a-8d11-87ea03db4101 req-232e276a-f5c7-4c7d-b605-27e7a851600d tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0826132 May 20 04:28:57.285074 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f681fd08-a9b7-418a-8d11-87ea03db4101 req-703ae9cd-0ca4-4a57-bfcf-2ee58ed765f4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0428615 May 20 04:28:57.508313 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-16274fe8-7fc6-4d25-b46d-5db2dc953d98 req-0e18cdaf-ef20-49e2-b87d-8aa702947dce tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 210 time: 0.0490041 May 20 04:28:57.697208 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-df5d4b40-a777-415b-b6cb-b5dba053188a req-28aacc9e-bae3-4462-820c-c74af4af9c9a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:57.698608 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-df5d4b40-a777-415b-b6cb-b5dba053188a req-28aacc9e-bae3-4462-820c-c74af4af9c9a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 932 time: 0.0865424 May 20 04:28:57.743612 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-df5d4b40-a777-415b-b6cb-b5dba053188a req-bcf72c8f-f20c-4280-8722-a2fd5854b184 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0390546 May 20 04:28:57.874780 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device ca396585-cdb5-4a50-a4c9-d5954043208e up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:28:57.927063 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-82291c67-21a0-4c27-8798-46488cf3d7ee req-4ba711bc-fda1-425c-9072-ee5b85bb0d05 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:57.928893 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-82291c67-21a0-4c27-8798-46488cf3d7ee req-4ba711bc-fda1-425c-9072-ee5b85bb0d05 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 944 time: 0.0951908 May 20 04:28:57.961719 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port ca396585-cdb5-4a50-a4c9-d5954043208e completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:28:57.966437 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port ca396585-cdb5-4a50-a4c9-d5954043208e triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:28:57.967178 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:57.978797 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-82291c67-21a0-4c27-8798-46488cf3d7ee req-a70e1d7f-352f-4ce4-90f9-cd57317cc292 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0436776 May 20 04:28:58.093873 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:58.121245 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'ca396585-cdb5-4a50-a4c9-d5954043208e'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:28:58.131618 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port ca396585-cdb5-4a50-a4c9-d5954043208e, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=ca396585-cdb5-4a50-a4c9-d5954043208e,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:28:58.135443 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:28:58.138025 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:28:58.173663 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'ca396585-cdb5-4a50-a4c9-d5954043208e', 'code': 200} May 20 04:28:58.205622 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=ca396585-cdb5-4a50-a4c9-d5954043208e,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:28:58.208760 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:28:58.487571 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2f4397f6-2303-4894-9842-462cf00c9ae4 req-f548ac77-420d-4b19-98b0-40dbc570eb76 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:58.488941 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2f4397f6-2303-4894-9842-462cf00c9ae4 req-f548ac77-420d-4b19-98b0-40dbc570eb76 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0927703 May 20 04:28:58.554593 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2f4397f6-2303-4894-9842-462cf00c9ae4 req-e4d852b4-ef39-44a6-baeb-947559e3a276 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0594573 May 20 04:28:58.696606 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1193bc98-3fc2-437d-8571-d0d03d902763 req-faf4a518-4b1b-44d6-9830-9bfa425193f1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 210 time: 0.0455215 May 20 04:28:58.821555 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-93628057-0f23-443e-a940-176bab91cdbf req-c5e93760-8b35-4dce-83cf-8dc44c3026cf tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:58.822994 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-93628057-0f23-443e-a940-176bab91cdbf req-c5e93760-8b35-4dce-83cf-8dc44c3026cf tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.1250825 May 20 04:28:58.910748 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-93628057-0f23-443e-a940-176bab91cdbf req-6f6630c9-eb0b-42c3-ae91-3f4f69ec9ecd tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0805573 May 20 04:28:58.954969 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2fa7bb41-0dd4-4426-bd54-ccf13b23893d req-6a3ecd70-b7fc-4d48-a6ad-6d831821091e tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:58.957394 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2fa7bb41-0dd4-4426-bd54-ccf13b23893d req-6a3ecd70-b7fc-4d48-a6ad-6d831821091e tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 932 time: 0.0883293 May 20 04:28:59.008839 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2fa7bb41-0dd4-4426-bd54-ccf13b23893d req-7682ce72-822d-4363-9fec-0766efe25d8d tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0446806 May 20 04:28:59.190350 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ef519c7b-f784-4d8b-a5be-ecc72cd70dca req-bd888f54-1d3b-48fb-92d9-0d420c2a8230 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:59.192053 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ef519c7b-f784-4d8b-a5be-ecc72cd70dca req-bd888f54-1d3b-48fb-92d9-0d420c2a8230 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 945 time: 0.0857582 May 20 04:28:59.243437 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ef519c7b-f784-4d8b-a5be-ecc72cd70dca req-28d98b67-0ee8-47b3-b34e-96897f3d63c0 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0421376 May 20 04:28:59.776818 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d90f7116-ba26-4e3c-9648-e673bd7009ec req-2b541a7e-ebdb-49a2-838b-74d691eecb40 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:28:59.782362 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d90f7116-ba26-4e3c-9648-e673bd7009ec req-2b541a7e-ebdb-49a2-838b-74d691eecb40 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0884378 May 20 04:28:59.862610 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d90f7116-ba26-4e3c-9648-e673bd7009ec req-eb269f30-1a53-412d-9f70-0dd0b6dae0fb tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0729167 May 20 04:28:59.876964 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0fb648ac-5c70-4447-802e-c972c8d88aca req-4001ffce-d001-49f0-8678-d637761f3cbe tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 210 time: 0.0522954 May 20 04:29:00.106706 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6b6a589a-1214-430e-8ecd-55fcbd461ae7 req-ea97d157-fb0b-42cd-bf01-d5c3f45a3c29 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:00.108431 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6b6a589a-1214-430e-8ecd-55fcbd461ae7 req-ea97d157-fb0b-42cd-bf01-d5c3f45a3c29 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.0888286 May 20 04:29:00.200422 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6b6a589a-1214-430e-8ecd-55fcbd461ae7 req-812bc90d-4156-487a-987b-69c64a42bbc0 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0849648 May 20 04:29:00.235744 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-babec327-8549-442e-a4d0-997116d40bde req-20889283-35e8-4baa-8bd4-75983ea324c3 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:00.237184 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-babec327-8549-442e-a4d0-997116d40bde req-20889283-35e8-4baa-8bd4-75983ea324c3 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 932 time: 0.0999064 May 20 04:29:00.284021 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-babec327-8549-442e-a4d0-997116d40bde req-16bb8704-2099-4014-88be-f8a69b4c6b04 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0408025 May 20 04:29:00.446144 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75f78788-8924-471f-9b8a-6db5b8fc8ae4 req-d2237903-98eb-486b-86c4-0b2a4cff85ce tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:00.447675 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-75f78788-8924-471f-9b8a-6db5b8fc8ae4 req-d2237903-98eb-486b-86c4-0b2a4cff85ce tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 945 time: 0.0802379 May 20 04:29:00.497792 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-75f78788-8924-471f-9b8a-6db5b8fc8ae4 req-946d12c0-8fc4-4d24-81de-587f518a7996 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0441012 May 20 04:29:01.077726 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b65661ec-3af2-496e-a431-d4c179ccc332 req-bd914ef8-01ba-4d5a-9daf-d46d5a0531b8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 210 time: 0.0576885 May 20 04:29:01.118019 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-59b98a91-47a2-403e-a718-d750592c70f4 req-026cc856-d250-401d-8770-d97ac4ba015e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:01.119630 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-59b98a91-47a2-403e-a718-d750592c70f4 req-026cc856-d250-401d-8770-d97ac4ba015e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.1308846 May 20 04:29:01.251956 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-59b98a91-47a2-403e-a718-d750592c70f4 req-9aea7d7b-98bd-4311-a225-ac2e23653b33 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1257637 May 20 04:29:01.392355 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e2fc9efa-bb2f-41bd-8dc3-b80a37d67731 req-74318fcd-18a3-4b76-9adb-95855501ae1b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:01.393837 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e2fc9efa-bb2f-41bd-8dc3-b80a37d67731 req-74318fcd-18a3-4b76-9adb-95855501ae1b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.0856695 May 20 04:29:01.501885 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e2fc9efa-bb2f-41bd-8dc3-b80a37d67731 req-fe05645f-78d3-474c-9672-9f34d32207a2 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1016631 May 20 04:29:01.534874 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d2b16d29-32c4-4a2b-92b4-953cc9e52c38 req-40a48d27-9cc4-4355-a29d-cb35d7c6015b tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:01.536534 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d2b16d29-32c4-4a2b-92b4-953cc9e52c38 req-40a48d27-9cc4-4355-a29d-cb35d7c6015b tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 932 time: 0.1353066 May 20 04:29:01.581108 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d2b16d29-32c4-4a2b-92b4-953cc9e52c38 req-8deecf74-178d-4aae-b98b-dca50c830e37 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0387435 May 20 04:29:01.695806 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5f20354-e0a2-4b47-957b-1df93c6f1141 req-7784917c-26c3-48e5-8304-20ba7be8755f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:01.697231 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b5f20354-e0a2-4b47-957b-1df93c6f1141 req-7784917c-26c3-48e5-8304-20ba7be8755f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 945 time: 0.0829194 May 20 04:29:01.747456 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b5f20354-e0a2-4b47-957b-1df93c6f1141 req-cb862b45-fbbc-4ff6-8d7b-ad7859fdf55c tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0439806 May 20 04:29:02.251851 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fcdeaf28-de3a-49ab-b797-6c785c1dd784 req-24ffb07b-cb68-4e64-922b-c80a694d5abf tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 210 time: 0.0434022 May 20 04:29:02.446282 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4387d82c-56cb-4663-bfbf-f081baa82f5a req-3df79e1d-e261-4972-bb49-f5cb54c4aca4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:02.448495 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4387d82c-56cb-4663-bfbf-f081baa82f5a req-3df79e1d-e261-4972-bb49-f5cb54c4aca4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0860455 May 20 04:29:02.499063 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4387d82c-56cb-4663-bfbf-f081baa82f5a req-69c4d871-81cc-46b3-be26-979ed27567c6 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0438082 May 20 04:29:02.686896 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0412ff5-29d1-4f97-aacf-3e866188fe27 req-da901644-97b6-4bd6-a546-a8a1e0325b3a tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:02.688353 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d0412ff5-29d1-4f97-aacf-3e866188fe27 req-da901644-97b6-4bd6-a546-a8a1e0325b3a tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.0875351 May 20 04:29:02.787327 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d0412ff5-29d1-4f97-aacf-3e866188fe27 req-fdbcd99b-4efe-4334-a8a6-7fa78fc9ec49 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0639257 May 20 04:29:02.809967 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0230258b-9b06-4119-bf10-d3a3e2c501c1 req-17a90f9a-1ab8-403a-b81d-571cc861c17c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:02.809967 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0230258b-9b06-4119-bf10-d3a3e2c501c1 req-17a90f9a-1ab8-403a-b81d-571cc861c17c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 932 time: 0.1168277 May 20 04:29:02.852867 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0230258b-9b06-4119-bf10-d3a3e2c501c1 req-7bc2d68a-57fe-4126-b58c-707418a32819 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0397203 May 20 04:29:02.965761 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ae7b9ca-19bc-4fac-a813-972c2f57814b req-cc3b4933-2f1b-4c76-a7e6-1680e739ce7c tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:02.967645 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4ae7b9ca-19bc-4fac-a813-972c2f57814b req-cc3b4933-2f1b-4c76-a7e6-1680e739ce7c tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 945 time: 0.0845277 May 20 04:29:03.026134 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4ae7b9ca-19bc-4fac-a813-972c2f57814b req-92e9e98e-00cc-4160-885a-eb65920fda69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0493298 May 20 04:29:03.418524 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-34025e12-7863-438b-bcbe-e937e5069640 req-7febb82f-1eb2-4422-9ad1-0eb5f63a7b1b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 210 time: 0.0447583 May 20 04:29:03.704232 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-50eff3c6-85c0-4bec-8307-f279342d32f0 req-3cb2831c-ae59-4994-a272-4e68eb1d502e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:03.705563 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-50eff3c6-85c0-4bec-8307-f279342d32f0 req-3cb2831c-ae59-4994-a272-4e68eb1d502e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0874963 May 20 04:29:03.753703 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-50eff3c6-85c0-4bec-8307-f279342d32f0 req-54926aa0-6294-4bd1-b50e-714623a63ec3 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0414934 May 20 04:29:04.024279 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3700ecf8-c644-4f4e-bb3e-26dd2a2d0635 req-a00b0d67-7570-43c2-8946-22823b99872b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:04.025802 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3700ecf8-c644-4f4e-bb3e-26dd2a2d0635 req-a00b0d67-7570-43c2-8946-22823b99872b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.1286230 May 20 04:29:04.081176 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3700ecf8-c644-4f4e-bb3e-26dd2a2d0635 req-2d35952e-9161-4883-8170-80d2c3fcd06f tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0479848 May 20 04:29:04.101511 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-530f35ce-29ef-4e7e-a5d2-f134da4fe5cb req-25e69914-84dd-4bd8-99f7-7be41da943bf tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:04.103030 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-530f35ce-29ef-4e7e-a5d2-f134da4fe5cb req-25e69914-84dd-4bd8-99f7-7be41da943bf tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 932 time: 0.1347930 May 20 04:29:04.163534 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-530f35ce-29ef-4e7e-a5d2-f134da4fe5cb req-c6f1a401-d274-4b6e-ba48-bbd0a3da39ba tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0546470 May 20 04:29:04.273878 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0d2b8e1a-009e-4b9c-926e-c82615e0c575 req-6c0cf348-f2a6-4934-a6d7-c0f45f428f0c tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:04.275753 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0d2b8e1a-009e-4b9c-926e-c82615e0c575 req-6c0cf348-f2a6-4934-a6d7-c0f45f428f0c tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 945 time: 0.0823619 May 20 04:29:04.336557 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0d2b8e1a-009e-4b9c-926e-c82615e0c575 req-12f39c8b-d9b9-4c94-b4f7-a3eeaa5989d6 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0523999 May 20 04:29:04.583445 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cd4fcaef-9a4b-4dba-9278-dc138644c961 req-c075a01d-f2d7-417d-a372-f030ec6af600 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 210 time: 0.0424800 May 20 04:29:04.949026 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7874f2a2-645d-421c-a06f-40ecab90185a req-1712d528-d33b-4c30-bde0-c3b2902dcbd9 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:04.950386 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7874f2a2-645d-421c-a06f-40ecab90185a req-1712d528-d33b-4c30-bde0-c3b2902dcbd9 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0873940 May 20 04:29:04.995491 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7874f2a2-645d-421c-a06f-40ecab90185a req-4322811b-d422-466f-b19f-08b6af78f82a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0389590 May 20 04:29:05.277634 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f05a5bcc-5845-472b-9ca6-836ab1203322 req-165b724d-1e96-4a17-8956-36bcbc19a7de tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:05.279149 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f05a5bcc-5845-472b-9ca6-836ab1203322 req-165b724d-1e96-4a17-8956-36bcbc19a7de tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.0839062 May 20 04:29:05.379286 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f05a5bcc-5845-472b-9ca6-836ab1203322 req-d7573564-8469-4523-b0df-8e46694b8994 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0656579 May 20 04:29:05.406125 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6c001506-df11-485b-9f7d-b7c058ff730f req-ed4cb890-1505-41f8-ad66-87db73dbda73 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:05.408872 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6c001506-df11-485b-9f7d-b7c058ff730f req-ed4cb890-1505-41f8-ad66-87db73dbda73 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 932 time: 0.1280658 May 20 04:29:05.453811 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6c001506-df11-485b-9f7d-b7c058ff730f req-6e16e70c-c3ed-40fe-bc8b-2091e954647b tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0396340 May 20 04:29:05.609177 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:05.671340 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-40ce0fba-9408-4abb-9a2a-20c573503d7a req-5fd72007-7261-4e45-8d49-100035d34cef tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:05.690399 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-40ce0fba-9408-4abb-9a2a-20c573503d7a req-5fd72007-7261-4e45-8d49-100035d34cef tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 945 time: 0.1957040 May 20 04:29:05.774936 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-40ce0fba-9408-4abb-9a2a-20c573503d7a req-e230a05f-767f-4ed3-a2a6-053efbbae8ad tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0904884 May 20 04:29:05.779528 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-738bb191-ee4f-453c-86ed-74d718c94597 req-e12d175f-72a8-4779-8702-34d55d52f95a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 210 time: 0.0535245 May 20 04:29:06.196333 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d808cff6-c417-4bd9-a398-1e2d523ecde6 req-251a7616-554c-45f2-a679-35044f0eb5be tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:06.197791 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d808cff6-c417-4bd9-a398-1e2d523ecde6 req-251a7616-554c-45f2-a679-35044f0eb5be tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0827379 May 20 04:29:06.243127 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d808cff6-c417-4bd9-a398-1e2d523ecde6 req-275075f9-1282-430e-9e2a-cf16d11c87eb tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0395243 May 20 04:29:06.609125 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d099772a-06d0-4eb2-bba6-c26828b41276 req-f8998643-835f-4999-9291-ba80bb1a886d tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:06.610534 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d099772a-06d0-4eb2-bba6-c26828b41276 req-f8998643-835f-4999-9291-ba80bb1a886d tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.1174316 May 20 04:29:06.667890 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d099772a-06d0-4eb2-bba6-c26828b41276 req-51a45004-a149-4202-9fe8-2780ee8334b9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0508864 May 20 04:29:06.686923 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e5042930-0dcb-4fde-abdc-851c4414d4c8 req-c267a13b-1536-466b-b832-633acbb51221 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:06.688360 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e5042930-0dcb-4fde-abdc-851c4414d4c8 req-c267a13b-1536-466b-b832-633acbb51221 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 932 time: 0.1245799 May 20 04:29:06.736752 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e5042930-0dcb-4fde-abdc-851c4414d4c8 req-e9cb1f73-cf68-461d-8f13-0af369361d46 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0425642 May 20 04:29:06.973981 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1ba89dd8-713c-4652-a0b7-b4b47f5945d2 req-151432e0-99de-4cba-970e-b9f0c93a8c9a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 210 time: 0.0459540 May 20 04:29:07.018975 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e69167fd-6b64-4472-b44d-1622d974bc13 req-db79b3da-a272-488a-b207-da7a2df8ff81 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:07.020781 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e69167fd-6b64-4472-b44d-1622d974bc13 req-db79b3da-a272-488a-b207-da7a2df8ff81 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 945 time: 0.1346698 May 20 04:29:07.068386 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e69167fd-6b64-4472-b44d-1622d974bc13 req-04000fd7-38b3-4892-8eea-67dc4b1ac6e1 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0419741 May 20 04:29:07.454723 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-32f62eb0-767d-4735-9dbf-73ea35034ebe req-f4ae99e2-171e-4e0d-aed2-2ecd9c375a6d tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:07.456198 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-32f62eb0-767d-4735-9dbf-73ea35034ebe req-f4ae99e2-171e-4e0d-aed2-2ecd9c375a6d tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0954084 May 20 04:29:07.504271 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-32f62eb0-767d-4735-9dbf-73ea35034ebe req-64648e9a-e630-4a88-b9a4-7261ee7356a9 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0397034 May 20 04:29:07.912592 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-db5e67cb-939d-4a96-a104-da93b6f61ef5 req-7daae10a-5205-4283-b9fb-2857b1715951 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:07.913979 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-db5e67cb-939d-4a96-a104-da93b6f61ef5 req-7daae10a-5205-4283-b9fb-2857b1715951 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.1310055 May 20 04:29:07.966590 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-db5e67cb-939d-4a96-a104-da93b6f61ef5 req-bd75a5f5-19ed-4cf7-a81e-1b3d5361e022 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0454147 May 20 04:29:07.976728 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38fd919c-8225-44bd-b60e-11a5ccf3ac6a req-9c3339ca-d2b7-46f1-90d5-e2954449cfd0 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:07.978096 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-38fd919c-8225-44bd-b60e-11a5ccf3ac6a req-9c3339ca-d2b7-46f1-90d5-e2954449cfd0 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 932 time: 0.1253939 May 20 04:29:08.027432 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-38fd919c-8225-44bd-b60e-11a5ccf3ac6a req-bce78db7-719a-4220-bd39-b46779f156f1 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0424597 May 20 04:29:08.129812 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-20d2ce44-8be4-44b2-a0f6-fd0dbdbbd4e0 req-99bf9e3b-64c0-4cf5-b7a2-00fead7f1d8f tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 210 time: 0.0421879 May 20 04:29:08.253542 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-517fc713-14bc-458e-a018-a3911d27024c req-4376aaa0-6ac3-45db-a11e-c763b55eaf54 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:08.254850 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-517fc713-14bc-458e-a018-a3911d27024c req-4376aaa0-6ac3-45db-a11e-c763b55eaf54 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 945 time: 0.0809982 May 20 04:29:08.302234 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-517fc713-14bc-458e-a018-a3911d27024c req-38ff36aa-d180-43dc-9863-3772ad1be760 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0415137 May 20 04:29:08.714593 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1f2fb258-333b-4c7e-ac71-387c66ba4a1e req-fc1fa419-7c25-4dfc-aa45-4852eda83b3a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:08.716068 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1f2fb258-333b-4c7e-ac71-387c66ba4a1e req-fc1fa419-7c25-4dfc-aa45-4852eda83b3a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0836225 May 20 04:29:08.760953 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1f2fb258-333b-4c7e-ac71-387c66ba4a1e req-651e732b-02c6-4484-a52a-69aa478f0310 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0387814 May 20 04:29:09.226055 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cea831e8-81b4-434d-b1ef-6ffc520b2f96 req-4d74048d-7ff5-4c98-b573-41364bbb432e tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:09.227517 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cea831e8-81b4-434d-b1ef-6ffc520b2f96 req-4d74048d-7ff5-4c98-b573-41364bbb432e tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.1396661 May 20 04:29:09.239253 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e916efd1-caee-422d-bdcf-310fe070b640 req-b3087ddc-f4f4-4342-954f-e6514896773f tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:09.240575 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e916efd1-caee-422d-bdcf-310fe070b640 req-b3087ddc-f4f4-4342-954f-e6514896773f tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 932 time: 0.1084242 May 20 04:29:09.309813 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cea831e8-81b4-434d-b1ef-6ffc520b2f96 req-804392e7-f64d-4218-b84d-18706eee5679 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0683250 May 20 04:29:09.314139 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e916efd1-caee-422d-bdcf-310fe070b640 req-52e78428-b86e-4841-a5b0-69298078ee2c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0600398 May 20 04:29:09.490819 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b0181312-bc30-4520-80b2-b3d6a810a23d req-6090e8f9-216d-483f-8408-c2a5d7df9646 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 210 time: 0.0813615 May 20 04:29:09.532216 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d6aafa59-ec1d-4862-b98f-35a9c05709cf req-49053e4d-bdbc-4da8-bd6b-34a5286fb4bc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:09.533636 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d6aafa59-ec1d-4862-b98f-35a9c05709cf req-49053e4d-bdbc-4da8-bd6b-34a5286fb4bc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 945 time: 0.0909672 May 20 04:29:09.581014 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d6aafa59-ec1d-4862-b98f-35a9c05709cf req-d1adb0e2-7d0c-4d1c-8299-e2910644e588 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0414228 May 20 04:29:09.961722 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-aa8f3bba-daeb-41dc-a008-cdb604de1893 req-fc2899a1-cd0a-4109-989e-7b73d39e5141 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:09.963253 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-aa8f3bba-daeb-41dc-a008-cdb604de1893 req-fc2899a1-cd0a-4109-989e-7b73d39e5141 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0789330 May 20 04:29:10.008896 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-aa8f3bba-daeb-41dc-a008-cdb604de1893 req-53f9131d-ed86-48b5-b609-09e0ab7e6638 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0384307 May 20 04:29:10.219626 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Request body: {'port': {'device_id': 'dfc47d7d-eb29-4bb6-a680-660af78ffcf3', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:29:10.221228 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-9770e281-b7d1-4c65-8a54-43975627d9d4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Request body: {'port': {'device_id': '74d085a4-f817-48d6-b8d8-8504cc4ca99d', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'admin_state_up': True, 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'security_groups': ['7a1d2a6b-5b70-4cb9-a737-fa50141ab0be']}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:29:10.415143 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-9770e281-b7d1-4c65-8a54-43975627d9d4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:29:10.425241 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-9770e281-b7d1-4c65-8a54-43975627d9d4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:29:10.546513 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-9770e281-b7d1-4c65-8a54-43975627d9d4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Made reservation on behalf of 9fafdf3a29024ac588e5b24719735677 for: {'port': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:29:10.549972 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-9770e281-b7d1-4c65-8a54-43975627d9d4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:10.556571 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:10.722588 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7dbe087b-8ebf-4bed-96d6-c7265aefe99a req-b2712be5-be11-44ef-86b9-82c8b89dcc8e tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:10.724062 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7dbe087b-8ebf-4bed-96d6-c7265aefe99a req-b2712be5-be11-44ef-86b9-82c8b89dcc8e tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 932 time: 0.2638817 May 20 04:29:10.737535 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e67db789-dde9-4f0c-9493-94c0cb4eebb1 req-72427e66-4900-400b-931a-e8decf8bf2e4 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:10.738438 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-306d595a-a0ff-41c3-8f09-e9567085686a req-90903f32-a5a1-44ce-b67a-443b58c94c0b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:10.738997 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e67db789-dde9-4f0c-9493-94c0cb4eebb1 req-72427e66-4900-400b-931a-e8decf8bf2e4 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=198bc8ab083f4a729f32b8ef27a6ee77&device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 1098 time: 0.3716872 May 20 04:29:10.739895 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-306d595a-a0ff-41c3-8f09-e9567085686a req-90903f32-a5a1-44ce-b67a-443b58c94c0b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.2460277 May 20 04:29:10.810888 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-245a5e09-b349-47e5-8195-fac86e91aa13 req-fe96eef2-4a64-4ee5-b5cf-63e83f10ec81 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:10.812259 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-245a5e09-b349-47e5-8195-fac86e91aa13 req-fe96eef2-4a64-4ee5-b5cf-63e83f10ec81 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 1098 time: 0.2473183 May 20 04:29:10.865782 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-9770e281-b7d1-4c65-8a54-43975627d9d4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Allocated IP 172.20.0.9 (64bcea8a-6d35-406b-a823-fa1185008174/203a3962-2282-4f51-ac92-c9695b5c0cfe/b1d353b0-0c40-42fa-a24f-0e337c3c1d7c) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:29:10.867921 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7dbe087b-8ebf-4bed-96d6-c7265aefe99a req-15197326-99fa-4c36-bf65-5e3cff8e4830 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1372662 May 20 04:29:10.870551 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-306d595a-a0ff-41c3-8f09-e9567085686a req-25356403-c6f9-466c-9cfa-380f4b209fbb tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1087320 May 20 04:29:10.951177 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-245a5e09-b349-47e5-8195-fac86e91aa13 req-dea6d56e-d29c-4cd2-9c4a-a5d646750026 service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0909073 May 20 04:29:10.982402 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e67db789-dde9-4f0c-9493-94c0cb4eebb1 req-58daa9e3-82c4-4fde-b9a4-9fd71de3258b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:10.984031 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e67db789-dde9-4f0c-9493-94c0cb4eebb1 req-58daa9e3-82c4-4fde-b9a4-9fd71de3258b service neutron] 10.222.0.74 "GET /v2.0/networks?id=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d HTTP/1.1" status: 200 len: 910 time: 0.2228787 May 20 04:29:11.122946 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-687c5062-b7b5-4781-b49e-eda94914efa7 req-79b982f0-765e-43f1-a70f-108d85d0e503 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 210 time: 0.0962710 May 20 04:29:11.134687 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 {{(pid=91643) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:29:11.151790 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Notify callbacks [] for port, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:11.172539 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.provisioning_blocks [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Transition to ACTIVE for port object 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 will not be triggered until provisioned by entity L2. {{(pid=91643) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:29:11.182378 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:11.187830 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:11.201622 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e67db789-dde9-4f0c-9493-94c0cb4eebb1 req-cda9e4f2-cae1-45d6-9e5e-cce55beb711a service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=2cf59c2c-723c-4450-8b98-0809b16290a8 HTTP/1.1" status: 200 len: 1001 time: 0.1326485 May 20 04:29:11.220519 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-9770e281-b7d1-4c65-8a54-43975627d9d4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:11.239771 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c0e69be1-9fad-42f3-975c-0d55f09bf08a req-455c7a86-12a9-4805-9e91-96fdc5bae7a2 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:11.241195 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c0e69be1-9fad-42f3-975c-0d55f09bf08a req-455c7a86-12a9-4805-9e91-96fdc5bae7a2 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 945 time: 0.1734924 May 20 04:29:11.280641 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-9770e281-b7d1-4c65-8a54-43975627d9d4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Transition to ACTIVE for port object b1d353b0-0c40-42fa-a24f-0e337c3c1d7c will not be triggered until provisioned by entity DHCP. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:29:11.292982 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e67db789-dde9-4f0c-9493-94c0cb4eebb1 req-6c0ba727-b710-4ab7-bb02-914d8e343bd3 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:11.294437 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e67db789-dde9-4f0c-9493-94c0cb4eebb1 req-6c0ba727-b710-4ab7-bb02-914d8e343bd3 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3 HTTP/1.1" status: 200 len: 848 time: 0.0708876 May 20 04:29:11.318735 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-9770e281-b7d1-4c65-8a54-43975627d9d4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:11.326793 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:11.388423 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Pushing event updated for resources: {'Port': ['ID=5f77cfe6-85e2-4a40-9a27-6ed37313ae13,revision_number=2']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:11.419788 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.232s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:11.434347 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-9770e281-b7d1-4c65-8a54-43975627d9d4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Network 64bcea8a-6d35-406b-a823-fa1185008174 is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:29:11.438444 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-9770e281-b7d1-4c65-8a54-43975627d9d4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:29:11.440958 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-9770e281-b7d1-4c65-8a54-43975627d9d4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:29:11.498991 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c0e69be1-9fad-42f3-975c-0d55f09bf08a req-e4d57052-ecab-4f59-a8a8-a76814310771 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.2459114 May 20 04:29:11.513392 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9770e281-b7d1-4c65-8a54-43975627d9d4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Pushing event updated for resources: {'Port': ['ID=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:11.516346 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-9770e281-b7d1-4c65-8a54-43975627d9d4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.190s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:11.522592 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-9770e281-b7d1-4c65-8a54-43975627d9d4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:11.523305 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-9770e281-b7d1-4c65-8a54-43975627d9d4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:11.551933 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-aa89a75b-4b30-4605-8c64-474ef0731738 req-aad946bc-27b0-4e9f-bdab-2653c3e49189 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:11.553590 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-aa89a75b-4b30-4605-8c64-474ef0731738 req-aad946bc-27b0-4e9f-bdab-2653c3e49189 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.3037691 May 20 04:29:11.558865 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Attempting to bind port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:29:11.559530 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Attempting to bind port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': 'd558b057-1c72-4efe-ab53-567b572e2de5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 801, 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e'}] {{(pid=91643) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:29:11.559684 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Attempting to bind port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 on network 9798d104-d4de-413f-b41d-fb9d7e8a818e {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:29:11.563337 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-9770e281-b7d1-4c65-8a54-43975627d9d4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.3465269 May 20 04:29:11.569200 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e67db789-dde9-4f0c-9493-94c0cb4eebb1 req-7bee19e0-cdb3-449d-ade5-996fe68a9f3d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:11.570683 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e67db789-dde9-4f0c-9493-94c0cb4eebb1 req-7bee19e0-cdb3-449d-ade5-996fe68a9f3d service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1988373 May 20 04:29:11.576519 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Request body: {'port': {'device_id': '74d085a4-f817-48d6-b8d8-8504cc4ca99d', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:29:11.579703 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 on network 9798d104-d4de-413f-b41d-fb9d7e8a818e not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:29:11.580025 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Attempting to bind port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 on network 9798d104-d4de-413f-b41d-fb9d7e8a818e {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:29:11.591301 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 29, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:29:11.591675 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Checking segment: {'id': 'd558b057-1c72-4efe-ab53-567b572e2de5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 801, 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91643) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:29:11.592355 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Bound using segment: {'id': 'd558b057-1c72-4efe-ab53-567b572e2de5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 801, 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e'} {{(pid=91643) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:29:11.592836 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.managers [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Bound port: 5f77cfe6-85e2-4a40-9a27-6ed37313ae13, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'd558b057-1c72-4efe-ab53-567b572e2de5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 801, 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e'}}] {{(pid=91643) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:29:11.593119 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:11.662270 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-aa89a75b-4b30-4605-8c64-474ef0731738 req-654f0628-ae5e-4291-b012-8212ae89ef3e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0998650 May 20 04:29:11.701244 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-6a114e85-29bb-43f3-9275-8f20c57ab629 None None] Provisioning for port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:29:11.703438 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] For port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13, host hv-cinder-78240, cleared binding levels {{(pid=91643) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:29:11.706595 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-6a114e85-29bb-43f3-9275-8f20c57ab629 None None] Provisioning complete for port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:29:11.706914 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] For port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=5f77cfe6-85e2-4a40-9a27-6ed37313ae13,segment=NetworkSegment(d558b057-1c72-4efe-ab53-567b572e2de5),segment_id=d558b057-1c72-4efe-ab53-567b572e2de5)] {{(pid=91643) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:29:11.707655 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-6a114e85-29bb-43f3-9275-8f20c57ab629 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:11.721947 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:11.732531 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-87845b71-f488-4afc-951f-c639c2e6856c service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:11.764724 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-6a114e85-29bb-43f3-9275-8f20c57ab629 None None] Port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:29:11.766611 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:11.767255 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:11.768977 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] 10.222.0.74 "PUT /v2.0/ports/5f77cfe6-85e2-4a40-9a27-6ed37313ae13 HTTP/1.1" status: 200 len: 1094 time: 1.5565124 May 20 04:29:11.775487 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'dfc47d7d-eb29-4bb6-a680-660af78ffcf3', 'tag': '5f77cfe6-85e2-4a40-9a27-6ed37313ae13'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:29:11.791291 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:11.812377 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e67db789-dde9-4f0c-9493-94c0cb4eebb1 req-9e24516e-6518-49e7-a537-f5c80315952a service neutron] 10.222.0.74 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2033970 May 20 04:29:11.816205 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Pushing event updated for resources: {'Port': ['ID=5f77cfe6-85e2-4a40-9a27-6ed37313ae13,revision_number=3']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:11.820021 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.088s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:11.938766 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-86dd7d3e-f3e9-4e90-b0cf-5d5a57f3bc2d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:11.940322 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-86dd7d3e-f3e9-4e90-b0cf-5d5a57f3bc2d service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=13d0e12c764f483ea457246b10c4cf7e&device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 1097 time: 0.1113691 May 20 04:29:12.017678 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'dfc47d7d-eb29-4bb6-a680-660af78ffcf3', 'tag': '5f77cfe6-85e2-4a40-9a27-6ed37313ae13', 'status': 'completed', 'code': 200} May 20 04:29:12.034650 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-2a10be5c-adbe-4c58-8590-9363e18ad3fd service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=5f77cfe6-85e2-4a40-9a27-6ed37313ae13 HTTP/1.1" status: 200 len: 217 time: 0.0821950 May 20 04:29:12.099795 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e67db789-dde9-4f0c-9493-94c0cb4eebb1 req-6e5ac837-a0e9-40a3-90e8-d7861b61704d service neutron] 10.222.0.74 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2512519 May 20 04:29:12.317798 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-5528aaef-82ac-4545-8006-eede229c7800 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:12.319250 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-5528aaef-82ac-4545-8006-eede229c7800 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=c5423164-3d24-4562-871f-ab03914420dc HTTP/1.1" status: 200 len: 852 time: 0.2392008 May 20 04:29:12.403317 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-908cd04a-046e-4e0b-b9bd-c1267d99bd1a req-5971367e-25f3-4042-9654-efe1003181dc tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:12.404778 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-908cd04a-046e-4e0b-b9bd-c1267d99bd1a req-5971367e-25f3-4042-9654-efe1003181dc tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.3656726 May 20 04:29:12.415099 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id b1d353b0-0c40-42fa-a24f-0e337c3c1d7c {{(pid=91644) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:29:12.415738 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0bc3e933-8401-4ef3-b410-d6296b669652 req-9b0f04cf-e227-48c9-bf18-50bc41d2f04e tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:12.417099 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0bc3e933-8401-4ef3-b410-d6296b669652 req-9b0f04cf-e227-48c9-bf18-50bc41d2f04e tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 944 time: 0.3060579 May 20 04:29:12.423051 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:12.443922 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Transition to ACTIVE for port object b1d353b0-0c40-42fa-a24f-0e337c3c1d7c will not be triggered until provisioned by entity L2. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:29:12.448713 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:12.454687 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d3f6b145-c977-4830-aafa-ec3a9046d930 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:12.501560 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-908cd04a-046e-4e0b-b9bd-c1267d99bd1a req-c5a3c0a9-7b04-4bd6-8eae-aa8cf28ee99f tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0827372 May 20 04:29:12.505816 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-f1dd6a4d-7436-473c-b6d3-8f162129984e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:12.507254 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-f1dd6a4d-7436-473c-b6d3-8f162129984e service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=9798d104-d4de-413f-b41d-fb9d7e8a818e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1787834 May 20 04:29:12.513560 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0bc3e933-8401-4ef3-b410-d6296b669652 req-34419d87-539d-497e-904d-ec9e35bec54d tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0772388 May 20 04:29:12.633641 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-5c5823e3-c008-4ba9-bc2a-e6fbe2562e23 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:12.719453 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Pushing event updated for resources: {'Port': ['ID=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:12.726377 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.272s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:12.772846 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Attempting to bind port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:29:12.773148 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Attempting to bind port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': '9d1b8515-c35d-48e6-86ad-4eff412e924a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 847, 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174'}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:29:12.773475 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Attempting to bind port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c on network 64bcea8a-6d35-406b-a823-fa1185008174 {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:29:12.778465 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c on network 64bcea8a-6d35-406b-a823-fa1185008174 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:29:12.778789 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Attempting to bind port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c on network 64bcea8a-6d35-406b-a823-fa1185008174 {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:29:12.784081 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 29, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:29:12.784414 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Checking segment: {'id': '9d1b8515-c35d-48e6-86ad-4eff412e924a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 847, 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:29:12.784791 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Bound using segment: {'id': '9d1b8515-c35d-48e6-86ad-4eff412e924a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 847, 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174'} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:29:12.785230 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Bound port: b1d353b0-0c40-42fa-a24f-0e337c3c1d7c, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '9d1b8515-c35d-48e6-86ad-4eff412e924a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 847, 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174'}}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:29:12.785551 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:12.821788 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-7b232a78-5ebc-4926-86ab-8b84755b0c07 service neutron] 10.222.0.74 "GET /v2.0/networks/9798d104-d4de-413f-b41d-fb9d7e8a818e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3065758 May 20 04:29:12.873795 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] For port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c, host hv-cinder-78240, cleared binding levels {{(pid=91644) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:29:12.876958 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] For port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91644) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:29:12.883484 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:12.887454 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-0e45daa8-d372-4095-893b-4904036cdd7d tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:12.928928 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0d57236-c4a3-42e0-b7f0-78b45afa86f6 req-c18a1869-c995-4cb0-92d6-68a9de2fda80 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:12.930326 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b0d57236-c4a3-42e0-b7f0-78b45afa86f6 req-c18a1869-c995-4cb0-92d6-68a9de2fda80 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.2551882 May 20 04:29:12.944276 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-36d51571-4947-4744-baed-54c8aa5e1678 req-caa78d41-d1d1-4676-bb3a-f2f362822166 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:12.946150 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-36d51571-4947-4744-baed-54c8aa5e1678 req-caa78d41-d1d1-4676-bb3a-f2f362822166 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 945 time: 0.2693729 May 20 04:29:12.950229 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:12.950835 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:12.952205 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] 10.222.0.74 "PUT /v2.0/ports/b1d353b0-0c40-42fa-a24f-0e337c3c1d7c HTTP/1.1" status: 200 len: 1093 time: 1.3798008 May 20 04:29:12.952702 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '74d085a4-f817-48d6-b8d8-8504cc4ca99d', 'tag': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:29:12.974120 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Pushing event updated for resources: {'Port': ['ID=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,revision_number=3']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:12.977210 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.090s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:13.054283 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b0d57236-c4a3-42e0-b7f0-78b45afa86f6 req-90a65999-44f2-40ad-817a-fce98c724eec tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1076407 May 20 04:29:13.066488 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-36d51571-4947-4744-baed-54c8aa5e1678 req-e5ea9c30-10f9-47da-80d8-5dd36f5b564c tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0945261 May 20 04:29:13.086040 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c529ee04-6695-4bb8-9a97-cf4614e8aa4d req-8c2fbef2-87b0-4f08-b33b-6478db1805d2 service neutron] 10.222.0.74 "GET /v2.0/networks/9798d104-d4de-413f-b41d-fb9d7e8a818e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2544649 May 20 04:29:13.090251 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-220fd339-c571-4f55-abc6-8107c9100da1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:13.101029 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-220fd339-c571-4f55-abc6-8107c9100da1 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=9fafdf3a29024ac588e5b24719735677&device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 1096 time: 0.1016092 May 20 04:29:13.175867 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '74d085a4-f817-48d6-b8d8-8504cc4ca99d', 'tag': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c', 'status': 'completed', 'code': 200} May 20 04:29:13.194317 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-c5cd8baa-1baa-42be-b81f-a10c6a345feb service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c HTTP/1.1" status: 200 len: 217 time: 0.0792041 May 20 04:29:13.258592 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-2eca0504-d09b-43ea-85da-557e6ed42d9c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:13.259953 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-2eca0504-d09b-43ea-85da-557e6ed42d9c service neutron] 10.222.0.74 "GET /v2.0/subnets?id=203a3962-2282-4f51-ac92-c9695b5c0cfe HTTP/1.1" status: 200 len: 856 time: 0.0583386 May 20 04:29:13.321071 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6bdf966d-841b-4cd2-aab6-4af0f8d3a371 req-af63aff9-f7f0-428c-91d4-c43648538deb service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:13.322606 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6bdf966d-841b-4cd2-aab6-4af0f8d3a371 req-af63aff9-f7f0-428c-91d4-c43648538deb service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=13d0e12c764f483ea457246b10c4cf7e&device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 1097 time: 0.1814442 May 20 04:29:13.337150 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-0c38ff98-d5ad-4fc7-b813-4ac0fe890972 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:13.338682 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-0c38ff98-d5ad-4fc7-b813-4ac0fe890972 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=64bcea8a-6d35-406b-a823-fa1185008174&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0718620 May 20 04:29:13.454367 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29fece6d-9fd1-41ca-9dd7-ad8ddfaf8bf8 req-b251a7a9-6f9a-49fc-92ca-22ec02b96a3c tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:13.455723 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-29fece6d-9fd1-41ca-9dd7-ad8ddfaf8bf8 req-b251a7a9-6f9a-49fc-92ca-22ec02b96a3c tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.2498434 May 20 04:29:13.470327 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-59720d11-fc0c-486e-8ac0-91de03056b1f service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1247602 May 20 04:29:13.488821 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7d2e41b1-6d8a-4361-80d9-29104715762a req-40db792e-f19f-4dcf-bfae-67ef7d8bee2b tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:13.490233 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7d2e41b1-6d8a-4361-80d9-29104715762a req-40db792e-f19f-4dcf-bfae-67ef7d8bee2b tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 945 time: 0.2394934 May 20 04:29:13.552441 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6bdf966d-841b-4cd2-aab6-4af0f8d3a371 req-b2e8a0f1-ce90-4811-b45c-7694faa58c5f service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:13.553900 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6bdf966d-841b-4cd2-aab6-4af0f8d3a371 req-b2e8a0f1-ce90-4811-b45c-7694faa58c5f service neutron] 10.222.0.74 "GET /v2.0/networks?id=9798d104-d4de-413f-b41d-fb9d7e8a818e HTTP/1.1" status: 200 len: 914 time: 0.2234025 May 20 04:29:13.565619 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-29fece6d-9fd1-41ca-9dd7-ad8ddfaf8bf8 req-210ab38b-c692-46f1-9da0-e6d0aaf1747c tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1029999 May 20 04:29:13.613934 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e9896efa-6a15-4b83-8b60-9ce112114afe req-e09d6ea6-0c7b-4163-806d-bf40916f17a4 service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1347029 May 20 04:29:13.629697 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7d2e41b1-6d8a-4361-80d9-29104715762a req-a02872e0-4e76-4e03-8591-6d857f954d99 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1315312 May 20 04:29:13.631023 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6bdf966d-841b-4cd2-aab6-4af0f8d3a371 req-5f762a5e-0cbe-4d55-9037-5d7e841bc6fd service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=5f77cfe6-85e2-4a40-9a27-6ed37313ae13 HTTP/1.1" status: 200 len: 217 time: 0.0685840 May 20 04:29:13.653722 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 41768) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:29:13.750997 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6bdf966d-841b-4cd2-aab6-4af0f8d3a371 req-edcbf523-9515-4ad3-925c-0e632486758d service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:13.752440 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6bdf966d-841b-4cd2-aab6-4af0f8d3a371 req-edcbf523-9515-4ad3-925c-0e632486758d service neutron] 10.222.0.74 "GET /v2.0/subnets?id=c5423164-3d24-4562-871f-ab03914420dc HTTP/1.1" status: 200 len: 852 time: 0.1150038 May 20 04:29:13.808413 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6274e35d-9c18-490f-acf4-e941f6219b12 req-3611db9c-de93-42ae-8575-0f37ede14a62 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:13.809797 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-6274e35d-9c18-490f-acf4-e941f6219b12 req-3611db9c-de93-42ae-8575-0f37ede14a62 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=9fafdf3a29024ac588e5b24719735677&device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 1096 time: 0.1131933 May 20 04:29:13.811878 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-af727e84-20a2-40b5-a05c-c9a0918419f0 tempest-TestVolumeBootPattern-1668223768 tempest-TestVolumeBootPattern-1668223768-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:13.813234 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-af727e84-20a2-40b5-a05c-c9a0918419f0 tempest-TestVolumeBootPattern-1668223768 tempest-TestVolumeBootPattern-1668223768-project-admin] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 1093 time: 0.1586039 May 20 04:29:13.816152 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 41772) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:29:13.843290 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Request body: {'floatingip': {'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'port_id': 'ca396585-cdb5-4a50-a4c9-d5954043208e', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'fixed_ip_address': '172.20.0.5'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:29:13.852447 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:29:13.869531 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Usage tracker for resource:floatingip and tenant:0e3baa0ea24d42f3ab59e40845b350fd is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:29:13.875294 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Quota usage for floatingip was recalculated. Used quota:1. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:29:13.882555 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-480b82d1-9556-4276-9964-13c236fd79ed req-0c0e3d63-29b1-4b7c-83c2-4d3853a99b85 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:13.883831 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-480b82d1-9556-4276-9964-13c236fd79ed req-0c0e3d63-29b1-4b7c-83c2-4d3853a99b85 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.2103963 May 20 04:29:13.927924 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attempting to reserve 1 items for resource floatingip. Total usage: 1; quota limit: 50; headroom:49 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:29:13.947519 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-404274a4-f655-43c2-a8be-4b28d106abbe req-9007dafd-9896-4ea6-af9c-96bf4f6a5cc2 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:13.948993 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-404274a4-f655-43c2-a8be-4b28d106abbe req-9007dafd-9896-4ea6-af9c-96bf4f6a5cc2 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 944 time: 0.2003841 May 20 04:29:13.956691 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6bdf966d-841b-4cd2-aab6-4af0f8d3a371 req-4ebf62fa-8e52-47e6-8511-d0344409d799 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:13.958070 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6bdf966d-841b-4cd2-aab6-4af0f8d3a371 req-4ebf62fa-8e52-47e6-8511-d0344409d799 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=9798d104-d4de-413f-b41d-fb9d7e8a818e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1732166 May 20 04:29:13.958774 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for floatingip, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:13.962200 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-480b82d1-9556-4276-9964-13c236fd79ed req-79b743e0-8f1d-49d4-8a36-c829b6ae0c57 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0694718 May 20 04:29:14.019264 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6274e35d-9c18-490f-acf4-e941f6219b12 req-71ae6676-8c2d-43a5-a877-96a281caa84f service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:14.020866 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-6274e35d-9c18-490f-acf4-e941f6219b12 req-71ae6676-8c2d-43a5-a877-96a281caa84f service neutron] 10.222.0.74 "GET /v2.0/networks?id=64bcea8a-6d35-406b-a823-fa1185008174 HTTP/1.1" status: 200 len: 918 time: 0.2022793 May 20 04:29:14.080905 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-404274a4-f655-43c2-a8be-4b28d106abbe req-c6f2d2ee-660d-4a1b-886d-536445b140ca tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1218798 May 20 04:29:14.123084 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6bdf966d-841b-4cd2-aab6-4af0f8d3a371 req-8166119a-cbd1-4cbb-86cb-87bf744b333f service neutron] 10.222.0.74 "GET /v2.0/networks/9798d104-d4de-413f-b41d-fb9d7e8a818e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1479025 May 20 04:29:14.257571 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-6274e35d-9c18-490f-acf4-e941f6219b12 req-31ffc71b-7452-4cf3-8333-ac35487dce53 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c HTTP/1.1" status: 200 len: 217 time: 0.2289038 May 20 04:29:14.342304 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6274e35d-9c18-490f-acf4-e941f6219b12 req-40fe2989-e6ed-48bf-adc2-77692a0a1ebe service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:14.343753 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-6274e35d-9c18-490f-acf4-e941f6219b12 req-40fe2989-e6ed-48bf-adc2-77692a0a1ebe service neutron] 10.222.0.74 "GET /v2.0/subnets?id=203a3962-2282-4f51-ac92-c9695b5c0cfe HTTP/1.1" status: 200 len: 856 time: 0.0782685 May 20 04:29:14.410945 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:14.421583 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6bdf966d-841b-4cd2-aab6-4af0f8d3a371 req-4ece3726-e2a8-4d83-a455-3f98b92318f3 service neutron] 10.222.0.74 "GET /v2.0/networks/9798d104-d4de-413f-b41d-fb9d7e8a818e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2911525 May 20 04:29:14.436990 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cb17cb74-42c2-402a-bd0d-1eb6237aa063 req-cf6f4c3c-24a6-4eb8-959e-f438d15331f3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:14.438437 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cb17cb74-42c2-402a-bd0d-1eb6237aa063 req-cf6f4c3c-24a6-4eb8-959e-f438d15331f3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.1623793 May 20 04:29:14.483295 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cb17cb74-42c2-402a-bd0d-1eb6237aa063 req-e8faa5f3-61e2-42a7-80e7-7f0bd6cbbe70 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0390182 May 20 04:29:14.488988 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6274e35d-9c18-490f-acf4-e941f6219b12 req-269360ea-4a81-45b0-9b9d-66b5e76ee232 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:14.490454 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-6274e35d-9c18-490f-acf4-e941f6219b12 req-269360ea-4a81-45b0-9b9d-66b5e76ee232 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=64bcea8a-6d35-406b-a823-fa1185008174&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1388144 May 20 04:29:14.657704 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Allocated IP 172.27.0.168 (712ce8c4-8feb-4719-bf13-e06992125cd9/e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5/c99984e7-2cf8-48d4-b093-fca7f6aef3ac) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:29:14.726907 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-6274e35d-9c18-490f-acf4-e941f6219b12 req-3d6fc65b-0bc0-4fdb-954a-3e02a0b2edc9 service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1983144 May 20 04:29:14.755865 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a833ee45-dcca-4339-8665-2d72ce1da550 req-d3e83993-f86f-4c99-9ae3-f53e7dd8e5b1 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:14.757296 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a833ee45-dcca-4339-8665-2d72ce1da550 req-d3e83993-f86f-4c99-9ae3-f53e7dd8e5b1 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0835552 May 20 04:29:14.801765 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a833ee45-dcca-4339-8665-2d72ce1da550 req-b8e0db0f-2640-43ef-a9fd-fda9affd3eb2 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0384400 May 20 04:29:14.959577 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:14.974782 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-6274e35d-9c18-490f-acf4-e941f6219b12 req-821c6370-61a9-4c9d-86d7-3796d8c437d6 service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1946824 May 20 04:29:14.974984 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:15.001646 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:15.005602 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:15.073468 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:29:15.085741 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:29:15.091500 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:29:15.095698 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-dda673fa-6327-4a09-b93b-7d99c2b42481 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:15.121131 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event updated for resources: {'Port': ['ID=c99984e7-2cf8-48d4-b093-fca7f6aef3ac,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:15.129520 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.124s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:15.165887 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-46f52c84-8efb-435f-af4f-3458d2547c23 req-c021e5bb-d100-4a77-8c85-48aceb3c2ed0 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:15.168930 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-46f52c84-8efb-435f-af4f-3458d2547c23 req-c021e5bb-d100-4a77-8c85-48aceb3c2ed0 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.0928316 May 20 04:29:15.257356 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-46f52c84-8efb-435f-af4f-3458d2547c23 req-395ed0e2-19fc-4ca3-a9f3-92668c75be04 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0822461 May 20 04:29:15.294098 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a16afd3c-01a8-400d-8e13-b3055fe4b688 req-3bee8480-b69e-4250-8e4c-35954433b5cf tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:15.295638 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a16afd3c-01a8-400d-8e13-b3055fe4b688 req-3bee8480-b69e-4250-8e4c-35954433b5cf tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 944 time: 0.0903304 May 20 04:29:15.341166 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a16afd3c-01a8-400d-8e13-b3055fe4b688 req-4bbb8b63-03c4-40c4-aa1c-ef773705edf0 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0390120 May 20 04:29:15.543961 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for floatingip, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:15.619079 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:15.684678 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-27702b12-b9be-46da-ba8c-831c3e77bf57 req-5204f75a-ed00-4980-8dc0-33d70af44825 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:15.686233 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-27702b12-b9be-46da-ba8c-831c3e77bf57 req-5204f75a-ed00-4980-8dc0-33d70af44825 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0920739 May 20 04:29:15.733001 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-27702b12-b9be-46da-ba8c-831c3e77bf57 req-167b85f5-1fc4-4619-85e9-3bf569006ff7 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0397661 May 20 04:29:15.822379 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:15.828123 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:15.832915 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:16.003044 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e7a3f509-3775-465a-9a6a-d67d9044da1d req-d2a97b63-09fa-4a52-b01b-4822cb73b3a4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:16.004657 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e7a3f509-3775-465a-9a6a-d67d9044da1d req-d2a97b63-09fa-4a52-b01b-4822cb73b3a4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0948000 May 20 04:29:16.054491 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e7a3f509-3775-465a-9a6a-d67d9044da1d req-9ad206c2-aa93-4cf9-9262-abaaa5b3c1ce tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0438623 May 20 04:29:16.055970 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event updated for resources: {'Port': ['ID=c99984e7-2cf8-48d4-b093-fca7f6aef3ac,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:16.060317 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.227s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:16.068185 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-8003251'] for floatingip, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:16.198415 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:16.246981 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.db.l3_db [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Floating IP f7e7609b-a65e-451e-8db6-b6e1bcbaa2e9 associated. External IP: 172.27.0.168, port: ca396585-cdb5-4a50-a4c9-d5954043208e. May 20 04:29:16.444071 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-905d8466-53aa-4a7b-9045-8431c25f4ff6 req-2f9dc4f4-24a9-49b8-9451-afa77cf90c75 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:16.445541 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-905d8466-53aa-4a7b-9045-8431c25f4ff6 req-2f9dc4f4-24a9-49b8-9451-afa77cf90c75 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.0815175 May 20 04:29:16.541039 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-905d8466-53aa-4a7b-9045-8431c25f4ff6 req-0d9a2d02-ab89-43c7-93fd-edd5c24953c2 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0619965 May 20 04:29:16.564974 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9cbba344-618b-4e37-92de-84cb35be36a0 req-b4199758-fedf-4771-83f6-2fe6df044eda tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:16.566425 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9cbba344-618b-4e37-92de-84cb35be36a0 req-b4199758-fedf-4771-83f6-2fe6df044eda tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 944 time: 0.1178136 May 20 04:29:16.611546 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9cbba344-618b-4e37-92de-84cb35be36a0 req-a791d1f4-3b95-4b1d-9cf4-ca2773a63f16 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0390520 May 20 04:29:16.634053 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Router 4969b258-f606-4def-b666-09764c475749 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91644) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:29:16.808789 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:29:16.819874 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-82951a93-2732-43d3-82c0-74d31a8176fd None None] Sync routers for ids ['4969b258-f606-4def-b666-09764c475749'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:29:16.832244 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-82951a93-2732-43d3-82c0-74d31a8176fd None None] Routers {'4969b258-f606-4def-b666-09764c475749'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:29:16.833043 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-82951a93-2732-43d3-82c0-74d31a8176fd None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 9, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 9, "log_agent_heartbeats": false, "routers": 9},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:29:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['4969b258-f606-4def-b666-09764c475749'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:29:16.835408 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:16.896012 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 996 time: 3.0785859 May 20 04:29:16.896611 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'tag': 'ca396585-cdb5-4a50-a4c9-d5954043208e'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:29:16.937737 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5b69ac1a-22e2-47ec-ae74-5908c4fe5541 req-cbc4d265-e044-46bf-b4a5-840df15533ac tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:16.939226 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5b69ac1a-22e2-47ec-ae74-5908c4fe5541 req-cbc4d265-e044-46bf-b4a5-840df15533ac tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0930326 May 20 04:29:16.944558 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'tag': 'ca396585-cdb5-4a50-a4c9-d5954043208e', 'status': 'completed', 'code': 200} May 20 04:29:16.988556 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5b69ac1a-22e2-47ec-ae74-5908c4fe5541 req-144bc7b4-5dad-47d9-8a75-fff14c66a6d0 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0425673 May 20 04:29:17.033314 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-82951a93-2732-43d3-82c0-74d31a8176fd None None] Set router 4969b258-f606-4def-b666-09764c475749 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:29:17.063443 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cffacfdd-0ccc-4756-95a3-fe5b9911e784 req-74bad180-bded-4fdd-bcfb-f4664fd9dfff service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:17.064914 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-cffacfdd-0ccc-4756-95a3-fe5b9911e784 req-74bad180-bded-4fdd-bcfb-f4664fd9dfff service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=0e3baa0ea24d42f3ab59e40845b350fd&device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 1098 time: 0.0921433 May 20 04:29:17.202744 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cffacfdd-0ccc-4756-95a3-fe5b9911e784 req-3be02797-6f3e-4069-995e-3a29081091cf service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:17.204145 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-cffacfdd-0ccc-4756-95a3-fe5b9911e784 req-3be02797-6f3e-4069-995e-3a29081091cf service neutron] 10.222.0.74 "GET /v2.0/networks?id=14c537db-01aa-4360-a348-47f94c75e03b HTTP/1.1" status: 200 len: 911 time: 0.1322114 May 20 04:29:17.259399 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-23250fea-d247-4d6a-bc14-bb0d0fa2fe66 req-ffdc0e07-e2ac-4320-9a9c-a430c380568b tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:17.264070 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-23250fea-d247-4d6a-bc14-bb0d0fa2fe66 req-ffdc0e07-e2ac-4320-9a9c-a430c380568b tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0942979 May 20 04:29:17.306398 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-23250fea-d247-4d6a-bc14-bb0d0fa2fe66 req-bff985e1-7eb5-452b-b102-6aa8a0be1a99 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0384376 May 20 04:29:17.314999 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-cffacfdd-0ccc-4756-95a3-fe5b9911e784 req-3fcadd0a-6148-444c-b638-c255e0896851 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=ca396585-cdb5-4a50-a4c9-d5954043208e HTTP/1.1" status: 200 len: 999 time: 0.1034687 May 20 04:29:17.383842 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cffacfdd-0ccc-4756-95a3-fe5b9911e784 req-2f564821-c164-47a7-9372-00d560e01d54 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:17.385267 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-cffacfdd-0ccc-4756-95a3-fe5b9911e784 req-2f564821-c164-47a7-9372-00d560e01d54 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=cdea475c-2929-4ae4-9271-87cc2fb9a378 HTTP/1.1" status: 200 len: 849 time: 0.0636065 May 20 04:29:17.391290 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-82951a93-2732-43d3-82c0-74d31a8176fd None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'f7e7609b-a65e-451e-8db6-b6e1bcbaa2e9', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'floating_ip_address': '172.27.0.168', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '4969b258-f606-4def-b666-09764c475749', 'port_id': 'ca396585-cdb5-4a50-a4c9-d5954043208e', 'fixed_ip_address': '172.20.0.5', 'status': 'DOWN', 'standard_attr_id': 569, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'mac_address': 'fa:16:3e:37:08:e4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:29:15Z', 'updated_at': '2021-05-20T04:29:15Z', 'revision_number': 0, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:50Z', 'revision_number': 5, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:29:17.395488 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-82951a93-2732-43d3-82c0-74d31a8176fd None None] Checking router: 4969b258-f606-4def-b666-09764c475749 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:29:17.404195 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-82951a93-2732-43d3-82c0-74d31a8176fd None None] Routers {'4969b258-f606-4def-b666-09764c475749'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:29:17.404799 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-82951a93-2732-43d3-82c0-74d31a8176fd None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 9, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 9, "log_agent_heartbeats": false, "routers": 9},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:29:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['4969b258-f606-4def-b666-09764c475749'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:29:17.470548 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cffacfdd-0ccc-4756-95a3-fe5b9911e784 req-82bd1d30-50f7-4196-8941-de1d48941e7d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:17.471963 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-cffacfdd-0ccc-4756-95a3-fe5b9911e784 req-82bd1d30-50f7-4196-8941-de1d48941e7d service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=14c537db-01aa-4360-a348-47f94c75e03b&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0790093 May 20 04:29:17.580964 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-82951a93-2732-43d3-82c0-74d31a8176fd None None] Set router 4969b258-f606-4def-b666-09764c475749 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:29:17.751567 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-cffacfdd-0ccc-4756-95a3-fe5b9911e784 req-8ee6cbbb-74b8-49a5-9d3a-fe2c08e4d330 service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2659061 May 20 04:29:17.781154 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bab3dfa2-9941-4ca6-8bd7-07ef7ccc5794 req-902f7acc-352b-43f9-8e41-304c7e092762 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:17.782962 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bab3dfa2-9941-4ca6-8bd7-07ef7ccc5794 req-902f7acc-352b-43f9-8e41-304c7e092762 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.1333237 May 20 04:29:17.839177 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bab3dfa2-9941-4ca6-8bd7-07ef7ccc5794 req-834354ad-1e3d-4b16-a859-681ebbec4acd tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0477712 May 20 04:29:17.855382 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dbcb9a41-867e-4ad5-8f25-e8976c1c6342 req-7861b2f9-4baf-4bdc-9762-d5270def1abd tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:17.857153 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-dbcb9a41-867e-4ad5-8f25-e8976c1c6342 req-7861b2f9-4baf-4bdc-9762-d5270def1abd tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 944 time: 0.1316190 May 20 04:29:17.899344 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-cffacfdd-0ccc-4756-95a3-fe5b9911e784 req-dbbba93a-b558-4ac2-8937-26ff8336d27f service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1410460 May 20 04:29:17.905573 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-dbcb9a41-867e-4ad5-8f25-e8976c1c6342 req-d7cd7e4d-7520-4f4c-ba60-528033e95643 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0422475 May 20 04:29:17.921135 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:17.942227 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b1d353b0-0c40-42fa-a24f-0e337c3c1d7c details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:29:18.118332 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-82951a93-2732-43d3-82c0-74d31a8176fd None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'f7e7609b-a65e-451e-8db6-b6e1bcbaa2e9', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'floating_ip_address': '172.27.0.168', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '4969b258-f606-4def-b666-09764c475749', 'port_id': 'ca396585-cdb5-4a50-a4c9-d5954043208e', 'fixed_ip_address': '172.20.0.5', 'status': 'DOWN', 'standard_attr_id': 569, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'mac_address': 'fa:16:3e:37:08:e4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:29:15Z', 'updated_at': '2021-05-20T04:29:15Z', 'revision_number': 0, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:50Z', 'revision_number': 5, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:29:18.127153 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:29:18.134470 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-82951a93-2732-43d3-82c0-74d31a8176fd None None] The sync data for ids ['4969b258-f606-4def-b666-09764c475749'] in dv-cinder-782405-9-iscsi is: [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'f7e7609b-a65e-451e-8db6-b6e1bcbaa2e9', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'floating_ip_address': '172.27.0.168', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '4969b258-f606-4def-b666-09764c475749', 'port_id': 'ca396585-cdb5-4a50-a4c9-d5954043208e', 'fixed_ip_address': '172.20.0.5', 'status': 'DOWN', 'standard_attr_id': 569, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'mac_address': 'fa:16:3e:37:08:e4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:29:15Z', 'updated_at': '2021-05-20T04:29:15Z', 'revision_number': 0, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:50Z', 'revision_number': 5, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:29:18.143459 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'port_id': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c', 'mac_address': 'fa:16:3e:cf:31:63', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 847, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.9'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:29:18.187393 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ea910914-c21d-4565-b36c-64493023b726 req-bed6cb43-5c7c-431a-842a-d582c5f04baf tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:18.188893 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ea910914-c21d-4565-b36c-64493023b726 req-bed6cb43-5c7c-431a-842a-d582c5f04baf tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0855107 May 20 04:29:18.210828 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:18.229368 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id b1d353b0-0c40-42fa-a24f-0e337c3c1d7c {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:29:18.237973 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ea910914-c21d-4565-b36c-64493023b726 req-d55cd476-a2c7-46ed-9b7f-fdf25d987a1c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0424988 May 20 04:29:18.245331 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:29:18.250012 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:18.250883 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:29:18.253128 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:18.400266 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:18.407740 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.154s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:18.453087 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c10ced47-d800-46a2-a16e-45362e96bb64 None None] New status for floating IP f7e7609b-a65e-451e-8db6-b6e1bcbaa2e9: ACTIVE {{(pid=91646) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 20 04:29:18.459116 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=5f77cfe6-85e2-4a40-9a27-6ed37313ae13,segment=NetworkSegment(d558b057-1c72-4efe-ab53-567b572e2de5),segment_id=d558b057-1c72-4efe-ab53-567b572e2de5)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:29:18.475928 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': '5f77cfe6-85e2-4a40-9a27-6ed37313ae13', 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e', 'port_id': '5f77cfe6-85e2-4a40-9a27-6ed37313ae13', 'mac_address': 'fa:16:3e:55:e5:39', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 801, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'c5423164-3d24-4562-871f-ab03914420dc', 'ip_address': '172.20.0.12'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:29:18.521338 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-24272128-99b6-4a41-9e94-719c51496b8c req-c18104e0-f3c2-457a-9b38-7370935963ad tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:18.522941 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-24272128-99b6-4a41-9e94-719c51496b8c req-c18104e0-f3c2-457a-9b38-7370935963ad tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0984700 May 20 04:29:18.535639 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:18.554053 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:29:18.575428 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=5f77cfe6-85e2-4a40-9a27-6ed37313ae13,segment=NetworkSegment(d558b057-1c72-4efe-ab53-567b572e2de5),segment_id=d558b057-1c72-4efe-ab53-567b572e2de5)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:29:18.580731 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-24272128-99b6-4a41-9e94-719c51496b8c req-db357764-73ec-4610-b6ee-4e8051c10ed4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0504320 May 20 04:29:18.584127 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:18.587144 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-925dbba8-f38d-4e26-af1f-c06367423c42 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:18.654704 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=5f77cfe6-85e2-4a40-9a27-6ed37313ae13,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:18.657847 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:18.963510 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['b1d353b0-0c40-42fa-a24f-0e337c3c1d7c'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:29:18.995912 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['5f77cfe6-85e2-4a40-9a27-6ed37313ae13'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:29:19.031392 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-72906e92-bc59-4f89-aed1-a9c3fb6e440b req-35f99307-763b-42ea-8949-7e6fbdab4145 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:19.032905 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-72906e92-bc59-4f89-aed1-a9c3fb6e440b req-35f99307-763b-42ea-8949-7e6fbdab4145 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.0832765 May 20 04:29:19.215060 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-72906e92-bc59-4f89-aed1-a9c3fb6e440b req-15f807e0-6f43-4caa-bfac-b2559355ce3a tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1505489 May 20 04:29:19.238757 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a1f305a7-0bca-41f7-a417-742bdf17cafe req-71e1ae6b-49ea-4e6b-9cde-63e8d454c7fe tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:19.240033 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a1f305a7-0bca-41f7-a417-742bdf17cafe req-71e1ae6b-49ea-4e6b-9cde-63e8d454c7fe tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 945 time: 0.2055340 May 20 04:29:19.292024 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a1f305a7-0bca-41f7-a417-742bdf17cafe req-fb8ebf2b-8f60-4793-b76b-7beb9ceed100 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0442793 May 20 04:29:19.446288 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3313f456-25d5-4f62-88bc-7fb4a5d203f9 req-aed24a29-99ff-4047-806d-153a2313a26f tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:19.447674 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3313f456-25d5-4f62-88bc-7fb4a5d203f9 req-aed24a29-99ff-4047-806d-153a2313a26f tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 944 time: 0.1009920 May 20 04:29:19.501281 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3313f456-25d5-4f62-88bc-7fb4a5d203f9 req-bc96d191-fd65-460a-9937-aa7aded4aae4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0466554 May 20 04:29:19.798378 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6d48df40-72af-4412-89fa-c5735d5e8986 req-0045957d-c476-4972-b542-f3daf9fb957d tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:19.799839 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6d48df40-72af-4412-89fa-c5735d5e8986 req-0045957d-c476-4972-b542-f3daf9fb957d tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0920482 May 20 04:29:19.853017 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6d48df40-72af-4412-89fa-c5735d5e8986 req-793d3113-54cf-4f23-af39-bdd4a765b281 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0470772 May 20 04:29:20.456671 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a50f3af1-15c7-4d7f-8dd0-732b9c301c67 req-85159ebc-8222-42bc-980c-7486a78f6bd2 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:20.458022 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a50f3af1-15c7-4d7f-8dd0-732b9c301c67 req-85159ebc-8222-42bc-980c-7486a78f6bd2 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.1297483 May 20 04:29:20.519063 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a50f3af1-15c7-4d7f-8dd0-732b9c301c67 req-77c16268-5151-44d1-97c3-617dce2e6faa tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0526884 May 20 04:29:20.536680 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7205b542-19e9-4b0a-b9d4-0f55b45bcc65 req-6d72c318-f907-4290-b2f9-101b31d33792 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:20.538162 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7205b542-19e9-4b0a-b9d4-0f55b45bcc65 req-6d72c318-f907-4290-b2f9-101b31d33792 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 945 time: 0.1404927 May 20 04:29:20.588561 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7205b542-19e9-4b0a-b9d4-0f55b45bcc65 req-65f77691-6c0b-4541-b059-f3d3d6dd1f98 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0424609 May 20 04:29:20.694259 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e8a3dbbc-74d4-491f-bbf8-6775cab0c348 req-03119895-63d4-4417-a662-d51edd5d042a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:20.695803 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e8a3dbbc-74d4-491f-bbf8-6775cab0c348 req-03119895-63d4-4417-a662-d51edd5d042a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 944 time: 0.0821161 May 20 04:29:20.743108 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e8a3dbbc-74d4-491f-bbf8-6775cab0c348 req-ebebbe6f-a320-4b61-8b03-d7f03b65e597 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0413523 May 20 04:29:21.060991 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a5ef2077-51ed-4e25-8e43-a1fc60e56f67 req-52504be7-b88d-47e8-9088-1905983bca53 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:21.062443 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a5ef2077-51ed-4e25-8e43-a1fc60e56f67 req-52504be7-b88d-47e8-9088-1905983bca53 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0929794 May 20 04:29:21.113352 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a5ef2077-51ed-4e25-8e43-a1fc60e56f67 req-9e8138b9-33e7-4e84-9855-f2a5815c2bf1 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0427985 May 20 04:29:21.459949 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-59ff67e0-181c-4180-ab03-b59346e7b5c1 req-01430837-e721-4aba-9eca-aa538fea0b20 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:21.461322 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-59ff67e0-181c-4180-ab03-b59346e7b5c1 req-01430837-e721-4aba-9eca-aa538fea0b20 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0853791 May 20 04:29:21.510360 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-59ff67e0-181c-4180-ab03-b59346e7b5c1 req-c51175db-2796-4766-adcd-a77f5d395413 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0428360 May 20 04:29:21.512319 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b8ac61c6-ad35-4bec-8e9b-e6767f85fdf4 req-b887eae5-b195-4941-9691-966fc5e9d2b9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:21.513967 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b8ac61c6-ad35-4bec-8e9b-e6767f85fdf4 req-b887eae5-b195-4941-9691-966fc5e9d2b9 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=198bc8ab083f4a729f32b8ef27a6ee77&device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 1098 time: 0.0748448 May 20 04:29:21.649263 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b8ac61c6-ad35-4bec-8e9b-e6767f85fdf4 req-eee8a445-3a37-46a2-8712-e3875d40d880 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:21.650698 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b8ac61c6-ad35-4bec-8e9b-e6767f85fdf4 req-eee8a445-3a37-46a2-8712-e3875d40d880 service neutron] 10.222.0.74 "GET /v2.0/networks?id=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d HTTP/1.1" status: 200 len: 910 time: 0.1304684 May 20 04:29:21.740626 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7622ef96-cba0-4ab5-aac8-eb292d901a38 req-5a13fc3b-08dd-4fa4-a4bc-b8adeaaf2190 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:21.741990 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7622ef96-cba0-4ab5-aac8-eb292d901a38 req-5a13fc3b-08dd-4fa4-a4bc-b8adeaaf2190 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.1177335 May 20 04:29:21.764366 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b8ac61c6-ad35-4bec-8e9b-e6767f85fdf4 req-a5c80d9c-b9b9-4417-a063-2b891a32f4f7 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=2cf59c2c-723c-4450-8b98-0809b16290a8 HTTP/1.1" status: 200 len: 1001 time: 0.1063998 May 20 04:29:21.805641 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7622ef96-cba0-4ab5-aac8-eb292d901a38 req-ac4e0cee-3ccf-44df-b3ac-17cf675e4c5e tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0574288 May 20 04:29:21.831308 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-53cb8ecc-c420-4a5d-be9e-a60898225241 req-5436626f-fa07-46df-8f1a-6d5e1ac6778d tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:21.832794 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-53cb8ecc-c420-4a5d-be9e-a60898225241 req-5436626f-fa07-46df-8f1a-6d5e1ac6778d tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 945 time: 0.1370404 May 20 04:29:21.834958 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b8ac61c6-ad35-4bec-8e9b-e6767f85fdf4 req-f908d98f-f8ad-4413-92c3-53312faf6484 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:21.836448 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b8ac61c6-ad35-4bec-8e9b-e6767f85fdf4 req-f908d98f-f8ad-4413-92c3-53312faf6484 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3 HTTP/1.1" status: 200 len: 848 time: 0.0658629 May 20 04:29:21.880498 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-53cb8ecc-c420-4a5d-be9e-a60898225241 req-596b08f3-eb00-4389-965d-7778a212a808 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0414152 May 20 04:29:21.926592 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b8ac61c6-ad35-4bec-8e9b-e6767f85fdf4 req-f0df85f9-0ca1-4e9a-84b3-bccbf85110f3 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:21.928108 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b8ac61c6-ad35-4bec-8e9b-e6767f85fdf4 req-f0df85f9-0ca1-4e9a-84b3-bccbf85110f3 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0851731 May 20 04:29:21.961913 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b1d353b0-0c40-42fa-a24f-0e337c3c1d7c up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:29:22.002589 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b91e5c85-9cb1-4392-a24b-3350d790816c req-d5a65a0a-fd9a-4f05-aeb7-af6fa6105ff6 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:22.003895 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b91e5c85-9cb1-4392-a24b-3350d790816c req-d5a65a0a-fd9a-4f05-aeb7-af6fa6105ff6 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 944 time: 0.0933321 May 20 04:29:22.034329 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:29:22.039752 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:29:22.040135 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:22.056652 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b91e5c85-9cb1-4392-a24b-3350d790816c req-496c3df2-04e3-4856-832f-a0a25a838b9c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0470152 May 20 04:29:22.072492 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b8ac61c6-ad35-4bec-8e9b-e6767f85fdf4 req-e3bdd711-250b-4a56-a95d-504fd546f103 service neutron] 10.222.0.74 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1364150 May 20 04:29:22.169482 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:22.199395 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '74d085a4-f817-48d6-b8d8-8504cc4ca99d', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:29:22.205146 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b8ac61c6-ad35-4bec-8e9b-e6767f85fdf4 req-cd4ece2c-c14b-49f9-9637-449b5c196bff service neutron] 10.222.0.74 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1255698 May 20 04:29:22.211038 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:29:22.215643 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:22.217103 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:29:22.243761 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-925dbba8-f38d-4e26-af1f-c06367423c42 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:22.285715 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '74d085a4-f817-48d6-b8d8-8504cc4ca99d', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c', 'code': 200} May 20 04:29:22.342710 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:22.347754 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:22.359627 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:29:22.364221 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:29:22.364596 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:22.486657 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:22.489131 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1cc1307c-3bd1-4fba-af9f-0e5eaea482ff req-29757fca-9ac0-4a4d-83bb-c563799cbd70 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:22.490604 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-1cc1307c-3bd1-4fba-af9f-0e5eaea482ff req-29757fca-9ac0-4a4d-83bb-c563799cbd70 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 1098 time: 0.0708747 May 20 04:29:22.520451 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=5f77cfe6-85e2-4a40-9a27-6ed37313ae13,segment=NetworkSegment(d558b057-1c72-4efe-ab53-567b572e2de5),segment_id=d558b057-1c72-4efe-ab53-567b572e2de5)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:29:22.525574 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:22.528199 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:22.535567 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-1cc1307c-3bd1-4fba-af9f-0e5eaea482ff req-20f8a6c5-e1d0-41c1-9a13-e2a972ce798b service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0368721 May 20 04:29:22.594016 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=5f77cfe6-85e2-4a40-9a27-6ed37313ae13,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:22.597087 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:22.736596 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9897283e-c474-4155-ad53-b2ee92b4a6c2 req-ca66c09a-5386-482f-8376-98ae404206b1 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:22.738049 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9897283e-c474-4155-ad53-b2ee92b4a6c2 req-ca66c09a-5386-482f-8376-98ae404206b1 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0954051 May 20 04:29:22.784368 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9897283e-c474-4155-ad53-b2ee92b4a6c2 req-211f84a1-4ead-4671-a2f9-230ee418cdd8 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0404656 May 20 04:29:23.063854 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5288fa09-1804-43cc-a23e-1c3d3fc46d60 req-f27b2d3d-dfea-4a77-ab4c-3ea26cd5277f tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:23.065647 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5288fa09-1804-43cc-a23e-1c3d3fc46d60 req-f27b2d3d-dfea-4a77-ab4c-3ea26cd5277f tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.1460769 May 20 04:29:23.130501 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-03246189-0647-40c5-9ed2-ad788c92b8b7 req-74515b7e-d16b-4d58-ba4a-ff8ca8e23760 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:23.131978 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-03246189-0647-40c5-9ed2-ad788c92b8b7 req-74515b7e-d16b-4d58-ba4a-ff8ca8e23760 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 946 time: 0.1439466 May 20 04:29:23.134712 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5288fa09-1804-43cc-a23e-1c3d3fc46d60 req-d64b3198-d56e-4522-ae08-45b436ee4195 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0627403 May 20 04:29:23.182658 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-03246189-0647-40c5-9ed2-ad788c92b8b7 req-8148e087-9762-464b-9217-c0675a3256c4 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0424392 May 20 04:29:23.310832 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5282d958-e571-44c3-8d95-705896c42ede req-c5883cc9-ca31-48d5-8797-9e6517d9d16f tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:23.312281 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5282d958-e571-44c3-8d95-705896c42ede req-c5883cc9-ca31-48d5-8797-9e6517d9d16f tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0833015 May 20 04:29:23.361432 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5282d958-e571-44c3-8d95-705896c42ede req-e61a16ec-ab2a-43da-96ba-f164043699cf tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0433593 May 20 04:29:23.980592 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-250df06d-55cd-4eda-8bc9-f4a2491b1394 req-2bf9da03-7179-4fd8-9348-b22370fb6aa4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:23.982076 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-250df06d-55cd-4eda-8bc9-f4a2491b1394 req-2bf9da03-7179-4fd8-9348-b22370fb6aa4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0863280 May 20 04:29:24.026820 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-250df06d-55cd-4eda-8bc9-f4a2491b1394 req-cd2ac40e-2655-4982-95ab-971795e03c5a tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0388510 May 20 04:29:24.287090 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'dfc47d7d-eb29-4bb6-a680-660af78ffcf3', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '5f77cfe6-85e2-4a40-9a27-6ed37313ae13'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:29:24.458019 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-64506cdf-0aa1-4cd3-bc08-5f0ab533ee14 req-eb53190f-9c3a-4575-97be-67e58ea91227 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:24.459377 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-64506cdf-0aa1-4cd3-bc08-5f0ab533ee14 req-eb53190f-9c3a-4575-97be-67e58ea91227 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.2158990 May 20 04:29:24.485165 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-77a9746f-00f9-4a34-8ef8-66d13b8102a5 req-3f0123b6-bce1-4dea-853e-1ab45c5ed61e tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:24.486541 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-77a9746f-00f9-4a34-8ef8-66d13b8102a5 req-3f0123b6-bce1-4dea-853e-1ab45c5ed61e tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 946 time: 0.1999743 May 20 04:29:24.541182 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-64506cdf-0aa1-4cd3-bc08-5f0ab533ee14 req-287ee61d-8e16-480c-b72c-4ef806bd7d61 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0756879 May 20 04:29:24.548198 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-77a9746f-00f9-4a34-8ef8-66d13b8102a5 req-bdceefc8-0c02-4448-8405-f534d8ed71ce tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0558596 May 20 04:29:24.592225 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'dfc47d7d-eb29-4bb6-a680-660af78ffcf3', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '5f77cfe6-85e2-4a40-9a27-6ed37313ae13', 'code': 200} May 20 04:29:24.731436 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d4330475-a5c2-42b0-8831-29df2c7d80c2 req-c23ac7d4-250d-430f-ae7a-58bb07776f5e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:24.732852 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d4330475-a5c2-42b0-8831-29df2c7d80c2 req-c23ac7d4-250d-430f-ae7a-58bb07776f5e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0831373 May 20 04:29:24.777020 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d4330475-a5c2-42b0-8831-29df2c7d80c2 req-98992f77-6418-4421-8596-b35075660de5 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0382059 May 20 04:29:24.819905 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e7feeb43-1ca4-48a4-baeb-19726b30f43a req-1e1e02da-a2f1-4eac-bbfc-5542cf6ce1ca service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:24.821391 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e7feeb43-1ca4-48a4-baeb-19726b30f43a req-1e1e02da-a2f1-4eac-bbfc-5542cf6ce1ca service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 1099 time: 0.0724916 May 20 04:29:24.862045 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-e7feeb43-1ca4-48a4-baeb-19726b30f43a req-bd04b225-d869-47b8-99a2-07358ccd0bc7 service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0343904 May 20 04:29:24.972577 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0919a943-eaa2-4a96-a60f-4f4df8b361c0 req-d652068a-70e1-4177-9416-6f80c7bc1bce tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:24.974007 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0919a943-eaa2-4a96-a60f-4f4df8b361c0 req-d652068a-70e1-4177-9416-6f80c7bc1bce tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0898759 May 20 04:29:24.985316 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 41860) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:29:24.990300 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-cc13c8bf-def7-491c-ba6b-ccf4d86e98fc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Request body: {'floatingip': {'port_id': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:29:25.065109 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-cc13c8bf-def7-491c-ba6b-ccf4d86e98fc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for floatingip, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:25.622264 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6965fa6a-2329-45b4-a67b-60a4b91f1734 req-62d645e4-40f0-40fb-8d8e-9289ddf41d30 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:25.624194 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-6965fa6a-2329-45b4-a67b-60a4b91f1734 req-62d645e4-40f0-40fb-8d8e-9289ddf41d30 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=39febf983bf24778a300f287dbeca829&device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 1098 time: 0.0775669 May 20 04:29:25.764897 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-cc13c8bf-def7-491c-ba6b-ccf4d86e98fc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for floatingip, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:25.765134 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6965fa6a-2329-45b4-a67b-60a4b91f1734 req-6fb6e0cd-9e20-4493-a4ed-6836b49e4310 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:25.766622 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-6965fa6a-2329-45b4-a67b-60a4b91f1734 req-6fb6e0cd-9e20-4493-a4ed-6836b49e4310 service neutron] 10.222.0.74 "GET /v2.0/networks?id=e57f6e29-4637-455e-af9d-0975c45c995c HTTP/1.1" status: 200 len: 909 time: 0.1355984 May 20 04:29:25.783836 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-cc13c8bf-def7-491c-ba6b-ccf4d86e98fc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-8003251'] for floatingip, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:25.869600 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-6965fa6a-2329-45b4-a67b-60a4b91f1734 req-cbb5378e-7490-42cc-9f0f-991d1fe9ff43 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=b858096a-44b8-4537-a167-0e9c03347445 HTTP/1.1" status: 200 len: 1001 time: 0.0958424 May 20 04:29:25.937182 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6965fa6a-2329-45b4-a67b-60a4b91f1734 req-01fa3574-5d9f-4c6a-b41f-3611a327d3bf service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:25.938618 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-6965fa6a-2329-45b4-a67b-60a4b91f1734 req-01fa3574-5d9f-4c6a-b41f-3611a327d3bf service neutron] 10.222.0.74 "GET /v2.0/subnets?id=49523a89-260b-4732-abff-507d74fed4c4 HTTP/1.1" status: 200 len: 847 time: 0.0600471 May 20 04:29:25.959563 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f3ebf873-d33a-49e0-9e54-b26efbb9b631 req-a590d29e-812c-4e24-a2c0-27d7508a8e87 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:25.961007 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f3ebf873-d33a-49e0-9e54-b26efbb9b631 req-a590d29e-812c-4e24-a2c0-27d7508a8e87 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 946 time: 0.2969413 May 20 04:29:25.977705 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8024f80d-0ffa-41aa-b179-fda9dbee4478 req-68598f2e-c15c-41bc-9498-a73ca738ec98 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:25.979129 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8024f80d-0ffa-41aa-b179-fda9dbee4478 req-68598f2e-c15c-41bc-9498-a73ca738ec98 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.3189383 May 20 04:29:26.025122 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6965fa6a-2329-45b4-a67b-60a4b91f1734 req-3f88df8a-1b4f-4412-aa91-bd5ece720cdf service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:26.026586 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-6965fa6a-2329-45b4-a67b-60a4b91f1734 req-3f88df8a-1b4f-4412-aa91-bd5ece720cdf service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=e57f6e29-4637-455e-af9d-0975c45c995c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0745490 May 20 04:29:26.049941 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.l3_db [None req-cc13c8bf-def7-491c-ba6b-ccf4d86e98fc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Floating IP 7134e5d9-58ed-4bdf-88ad-c3f9e36c5a1c associated. External IP: 172.27.0.70, port: b1d353b0-0c40-42fa-a24f-0e337c3c1d7c. May 20 04:29:26.230090 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f3ebf873-d33a-49e0-9e54-b26efbb9b631 req-d394f1d3-a306-40bb-a40c-94346f055d57 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2475829 May 20 04:29:26.232550 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8024f80d-0ffa-41aa-b179-fda9dbee4478 req-508cc3b7-155f-49b2-a1d2-4a160f02cd13 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.2447717 May 20 04:29:26.280839 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-6965fa6a-2329-45b4-a67b-60a4b91f1734 req-b70e8018-4c56-454b-8167-7e57c688225d service neutron] 10.222.0.74 "GET /v2.0/networks/e57f6e29-4637-455e-af9d-0975c45c995c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2456031 May 20 04:29:26.417378 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-6965fa6a-2329-45b4-a67b-60a4b91f1734 req-6fc034d7-4bd8-4a88-99b0-04a277a8814f service neutron] 10.222.0.74 "GET /v2.0/networks/e57f6e29-4637-455e-af9d-0975c45c995c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1249335 May 20 04:29:26.536292 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-cc13c8bf-def7-491c-ba6b-ccf4d86e98fc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91643) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:29:26.726901 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-cc13c8bf-def7-491c-ba6b-ccf4d86e98fc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:29:26.732382 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-cc13c8bf-def7-491c-ba6b-ccf4d86e98fc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:26.736801 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-df34144b-e89a-4193-a009-c455070a7c66 None None] Sync routers for ids ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:29:26.747931 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-df34144b-e89a-4193-a009-c455070a7c66 None None] Routers {'5a3f69b9-fd58-499f-bec9-75215cd7f8fc'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:29:26.748558 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-df34144b-e89a-4193-a009-c455070a7c66 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 9, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 9, "log_agent_heartbeats": false, "routers": 9},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:29:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:29:26.788725 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-cc13c8bf-def7-491c-ba6b-ccf4d86e98fc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "PUT /v2.0/floatingips/7134e5d9-58ed-4bdf-88ad-c3f9e36c5a1c HTTP/1.1" status: 200 len: 990 time: 1.8022454 May 20 04:29:26.789268 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '74d085a4-f817-48d6-b8d8-8504cc4ca99d', 'tag': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:29:26.833804 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '74d085a4-f817-48d6-b8d8-8504cc4ca99d', 'tag': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c', 'status': 'completed', 'code': 200} May 20 04:29:26.929618 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-98a53ac1-da09-460a-afb5-0c99cfb7da91 req-27281c21-0c6f-4fd6-b452-253ace4c812d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:26.931076 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-98a53ac1-da09-460a-afb5-0c99cfb7da91 req-27281c21-0c6f-4fd6-b452-253ace4c812d service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=9fafdf3a29024ac588e5b24719735677&device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 1098 time: 0.0701745 May 20 04:29:26.953751 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-df34144b-e89a-4193-a009-c455070a7c66 None None] Set router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:29:26.998782 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:29:27.073074 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-98a53ac1-da09-460a-afb5-0c99cfb7da91 req-0c97a98d-905b-48bc-a916-29059e76a57a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:27.074578 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-98a53ac1-da09-460a-afb5-0c99cfb7da91 req-0c97a98d-905b-48bc-a916-29059e76a57a service neutron] 10.222.0.74 "GET /v2.0/networks?id=64bcea8a-6d35-406b-a823-fa1185008174 HTTP/1.1" status: 200 len: 918 time: 0.1332929 May 20 04:29:27.186351 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-98a53ac1-da09-460a-afb5-0c99cfb7da91 req-6d59d279-dabd-4350-adf5-813d82427d9b service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c HTTP/1.1" status: 200 len: 998 time: 0.0987995 May 20 04:29:27.254954 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-98a53ac1-da09-460a-afb5-0c99cfb7da91 req-146ab3ab-2630-49e0-b58b-e319ce652bdb service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:27.256422 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-98a53ac1-da09-460a-afb5-0c99cfb7da91 req-146ab3ab-2630-49e0-b58b-e319ce652bdb service neutron] 10.222.0.74 "GET /v2.0/subnets?id=203a3962-2282-4f51-ac92-c9695b5c0cfe HTTP/1.1" status: 200 len: 856 time: 0.0634193 May 20 04:29:27.308926 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-df34144b-e89a-4193-a009-c455070a7c66 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port': {'id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:e4:9a:7c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:39Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': '7134e5d9-58ed-4bdf-88ad-c3f9e36c5a1c', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'floating_ip_address': '172.27.0.70', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'port_id': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c', 'fixed_ip_address': '172.20.0.9', 'status': 'DOWN', 'standard_attr_id': 565, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'mac_address': 'fa:16:3e:cf:31:63', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '74d085a4-f817-48d6-b8d8-8504cc4ca99d', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:28:50Z', 'updated_at': '2021-05-20T04:29:25Z', 'revision_number': 1, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'ac9495b7-5868-4104-85c9-3567f1350467', 'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'mac_address': 'fa:16:3e:12:5b:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_interface', 'standard_attr_id': 82, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:45Z', 'updated_at': '2021-05-20T04:21:55Z', 'revision_number': 5, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'subnets': [{'id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:29:27.313041 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-df34144b-e89a-4193-a009-c455070a7c66 None None] Checking router: 5a3f69b9-fd58-499f-bec9-75215cd7f8fc for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:29:27.322596 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-df34144b-e89a-4193-a009-c455070a7c66 None None] Routers {'5a3f69b9-fd58-499f-bec9-75215cd7f8fc'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:29:27.323283 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-df34144b-e89a-4193-a009-c455070a7c66 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 9, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 9, "log_agent_heartbeats": false, "routers": 9},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:29:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:29:27.349486 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-98a53ac1-da09-460a-afb5-0c99cfb7da91 req-eec36eac-8bf7-4498-bb69-35ac6c8decd3 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:27.351064 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-98a53ac1-da09-460a-afb5-0c99cfb7da91 req-eec36eac-8bf7-4498-bb69-35ac6c8decd3 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=64bcea8a-6d35-406b-a823-fa1185008174&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0840828 May 20 04:29:27.488687 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-98a53ac1-da09-460a-afb5-0c99cfb7da91 req-7437d65a-89d5-4ed8-82f4-ba776aea7f1d service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1286047 May 20 04:29:27.490158 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3bc7e4f6-ffe7-4774-8ae7-090792df45f1 req-971f15e8-9846-4a5a-8488-2796ac9af015 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:27.491641 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3bc7e4f6-ffe7-4774-8ae7-090792df45f1 req-971f15e8-9846-4a5a-8488-2796ac9af015 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 946 time: 0.1429694 May 20 04:29:27.499879 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ca1165e9-9126-4463-be38-5c6797028952 req-f4cf1ebe-cab4-4ce5-9966-6c13183f8c4e tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:27.501266 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ca1165e9-9126-4463-be38-5c6797028952 req-f4cf1ebe-cab4-4ce5-9966-6c13183f8c4e tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.1159253 May 20 04:29:27.510242 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-df34144b-e89a-4193-a009-c455070a7c66 None None] Set router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:29:27.568613 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3bc7e4f6-ffe7-4774-8ae7-090792df45f1 req-6f75753c-b117-4b93-a826-7d4ee6d734b8 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0663896 May 20 04:29:27.575749 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ca1165e9-9126-4463-be38-5c6797028952 req-d720543a-9615-4192-b790-dd0874107770 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0586586 May 20 04:29:27.630017 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-98a53ac1-da09-460a-afb5-0c99cfb7da91 req-be789091-3440-4b5e-b71f-aaaa2a8a8652 service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1320300 May 20 04:29:27.768041 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-11a509da-ca03-4905-8e0e-3c4e04d1f161 req-17804085-e16c-4332-8bd3-e035ccfb995a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:27.769536 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-11a509da-ca03-4905-8e0e-3c4e04d1f161 req-17804085-e16c-4332-8bd3-e035ccfb995a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 946 time: 0.0815229 May 20 04:29:27.817469 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-11a509da-ca03-4905-8e0e-3c4e04d1f161 req-fd8f12d9-2f00-47bd-ad1f-e9eb72c32bb6 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0420935 May 20 04:29:27.876386 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-df34144b-e89a-4193-a009-c455070a7c66 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port': {'id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:e4:9a:7c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:39Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': '7134e5d9-58ed-4bdf-88ad-c3f9e36c5a1c', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'floating_ip_address': '172.27.0.70', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'port_id': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c', 'fixed_ip_address': '172.20.0.9', 'status': 'DOWN', 'standard_attr_id': 565, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'mac_address': 'fa:16:3e:cf:31:63', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '74d085a4-f817-48d6-b8d8-8504cc4ca99d', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:28:50Z', 'updated_at': '2021-05-20T04:29:25Z', 'revision_number': 1, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'ac9495b7-5868-4104-85c9-3567f1350467', 'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'mac_address': 'fa:16:3e:12:5b:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_interface', 'standard_attr_id': 82, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:45Z', 'updated_at': '2021-05-20T04:21:55Z', 'revision_number': 5, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'subnets': [{'id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:29:27.880571 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-df34144b-e89a-4193-a009-c455070a7c66 None None] The sync data for ids ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'] in dv-cinder-782405-9-iscsi is: [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port': {'id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:e4:9a:7c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:39Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': '7134e5d9-58ed-4bdf-88ad-c3f9e36c5a1c', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'floating_ip_address': '172.27.0.70', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'port_id': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c', 'fixed_ip_address': '172.20.0.9', 'status': 'DOWN', 'standard_attr_id': 565, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'mac_address': 'fa:16:3e:cf:31:63', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '74d085a4-f817-48d6-b8d8-8504cc4ca99d', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:28:50Z', 'updated_at': '2021-05-20T04:29:25Z', 'revision_number': 1, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'ac9495b7-5868-4104-85c9-3567f1350467', 'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'mac_address': 'fa:16:3e:12:5b:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_interface', 'standard_attr_id': 82, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:45Z', 'updated_at': '2021-05-20T04:21:55Z', 'revision_number': 5, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'subnets': [{'id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:29:28.062451 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-f4412633-cee3-48ec-955d-db7b45613a97 None None] New status for floating IP 7134e5d9-58ed-4bdf-88ad-c3f9e36c5a1c: ACTIVE {{(pid=91646) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 20 04:29:28.220050 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-549436f0-ec65-442e-9a82-41599e2bc7db req-92444659-b864-4b23-9b73-09b744cfde0e tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:28.221514 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-549436f0-ec65-442e-9a82-41599e2bc7db req-92444659-b864-4b23-9b73-09b744cfde0e tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 946 time: 0.0813599 May 20 04:29:28.266016 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-549436f0-ec65-442e-9a82-41599e2bc7db req-4d9127b3-f62a-448c-8baf-0738de96bfe4 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0386181 May 20 04:29:28.794315 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ec29a2d1-d7b3-43f4-b6d4-ff1097a31bcb req-f4ed7c61-e0ed-421a-b252-f445ffe79f36 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:28.795778 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ec29a2d1-d7b3-43f4-b6d4-ff1097a31bcb req-f4ed7c61-e0ed-421a-b252-f445ffe79f36 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.0948505 May 20 04:29:28.844053 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ec29a2d1-d7b3-43f4-b6d4-ff1097a31bcb req-fe3892f4-c957-461e-8754-8d7ff96679a9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0393736 May 20 04:29:30.039551 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a61fe774-f6c6-48f0-8f1f-928c55acb215 req-df0c41e1-c12c-4b49-8447-0677e14cc692 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:30.041013 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a61fe774-f6c6-48f0-8f1f-928c55acb215 req-df0c41e1-c12c-4b49-8447-0677e14cc692 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.0829287 May 20 04:29:30.094199 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a61fe774-f6c6-48f0-8f1f-928c55acb215 req-d628b5a4-c63f-4d98-a9dd-f012a7572c9a tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0465291 May 20 04:29:30.365862 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 41922) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:29:30.518954 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-017d4798-09e9-4b25-8720-1b6152616e6f req-4e7369e4-f9d1-4a8f-99fe-3026f17e2907 service neutron] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff&fields=id&fields=resource_request HTTP/1.1" status: 200 len: 255 time: 0.1511703 May 20 04:29:30.574813 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a16a2a5f-c687-4f9e-80aa-ddff46836f3e req-1d15aa8a-836c-4a5c-940b-ac07d20209e4 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:30.576490 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a16a2a5f-c687-4f9e-80aa-ddff46836f3e req-1d15aa8a-836c-4a5c-940b-ac07d20209e4 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.0862732 May 20 04:29:30.626506 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a16a2a5f-c687-4f9e-80aa-ddff46836f3e req-e606195d-fd7e-43c6-9400-b1b9a2f7500c tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0444417 May 20 04:29:31.819912 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7e4a0ce4-79be-45eb-93a1-2d341fc0937a req-5389d536-5fd7-4750-821a-9c88051607a7 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:31.821342 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7e4a0ce4-79be-45eb-93a1-2d341fc0937a req-5389d536-5fd7-4750-821a-9c88051607a7 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.0841148 May 20 04:29:31.867818 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7e4a0ce4-79be-45eb-93a1-2d341fc0937a req-d770db6f-dcdc-4095-9ec4-477e38db31bb tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0405810 May 20 04:29:32.002717 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b858096a-44b8-4537-a167-0e9c03347445 no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:29:32.131665 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:32.164408 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '30d5e70e-9acd-400c-83be-01301f56a5ff', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': 'b858096a-44b8-4537-a167-0e9c03347445'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:29:32.173873 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b858096a-44b8-4537-a167-0e9c03347445, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b858096a-44b8-4537-a167-0e9c03347445,segment=NetworkSegment(c7d77442-5ac6-48cf-ab59-3f5962fc1e3e),segment_id=c7d77442-5ac6-48cf-ab59-3f5962fc1e3e)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:29:32.179988 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:32.182015 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:32.218570 dv-cinder-782405-9-iscsi neutron-server[91645]: WARNING neutron.notifiers.nova [-] Nova event: {'server_uuid': '30d5e70e-9acd-400c-83be-01301f56a5ff', 'name': 'network-vif-unplugged', 'status': 'failed', 'tag': 'b858096a-44b8-4537-a167-0e9c03347445', 'code': 422} returned with failed status May 20 04:29:32.250310 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=b858096a-44b8-4537-a167-0e9c03347445,revision_number=6']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:32.257725 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:32.258071 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:32.258399 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:32.258720 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:32.259039 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:32.259457 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:32.259827 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:32.260128 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:32.260464 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:32.260777 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:32.261087 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:32.261395 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:32.265330 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.083s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:33.083030 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a31d33f0-9f18-4155-a19d-d697178d3e72 req-3a5882cf-6950-4f04-be65-c2c473dd07a1 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:33.084528 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a31d33f0-9f18-4155-a19d-d697178d3e72 req-3a5882cf-6950-4f04-be65-c2c473dd07a1 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.0983863 May 20 04:29:33.135902 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a31d33f0-9f18-4155-a19d-d697178d3e72 req-914de5da-38f5-4fb8-a349-d0c4a72ccc88 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0421720 May 20 04:29:34.436506 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-50a481a1-dc90-46ba-b3e1-429b50ae4d4e req-4ebfab8e-428a-4bda-b3e8-cefd605f874f tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:34.437841 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-50a481a1-dc90-46ba-b3e1-429b50ae4d4e req-4ebfab8e-428a-4bda-b3e8-cefd605f874f tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.1873386 May 20 04:29:34.486627 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-50a481a1-dc90-46ba-b3e1-429b50ae4d4e req-0a7692f4-1235-4394-9b23-6f73fc15717b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0430679 May 20 04:29:35.600841 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:35.701623 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7789f122-c5d1-477f-a4e3-ae0f176859b9 req-16d1e94c-755f-4ecc-b136-6f96d2d1191a tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:35.703063 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7789f122-c5d1-477f-a4e3-ae0f176859b9 req-16d1e94c-755f-4ecc-b136-6f96d2d1191a tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.0920522 May 20 04:29:35.749515 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7789f122-c5d1-477f-a4e3-ae0f176859b9 req-5ac2ecdb-6376-4d9c-8241-156078dd2ec4 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0404758 May 20 04:29:36.955767 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c1a31aae-0b92-43ee-a8bc-e92660ed4dc5 req-ecdce00e-77c6-49e7-8677-e52e998338f2 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:36.957207 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c1a31aae-0b92-43ee-a8bc-e92660ed4dc5 req-ecdce00e-77c6-49e7-8677-e52e998338f2 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.0938437 May 20 04:29:37.013309 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c1a31aae-0b92-43ee-a8bc-e92660ed4dc5 req-cb00c0a9-d05b-477e-9de1-751385572c34 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0498190 May 20 04:29:38.207702 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-70d3eb1b-84bc-4aed-9b17-e1173473a6c0 req-822ed9e2-84c6-4b62-8408-feb1dbfdfd85 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:38.209153 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-70d3eb1b-84bc-4aed-9b17-e1173473a6c0 req-822ed9e2-84c6-4b62-8408-feb1dbfdfd85 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.0815048 May 20 04:29:38.256930 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-70d3eb1b-84bc-4aed-9b17-e1173473a6c0 req-cd81cd25-a0bb-4a9e-a523-0f7a71675d74 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0414813 May 20 04:29:38.686303 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1ac8ba14-123d-4564-9048-054d97e348fb req-1eaa9811-0e00-4b80-ae19-73960de090e5 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:38.687663 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1ac8ba14-123d-4564-9048-054d97e348fb req-1eaa9811-0e00-4b80-ae19-73960de090e5 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 946 time: 0.0890822 May 20 04:29:38.735878 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1ac8ba14-123d-4564-9048-054d97e348fb req-0ab188b3-731f-4d5c-901a-ce71ee006f71 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0394444 May 20 04:29:38.759327 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 41988) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:29:38.935621 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-017d4798-09e9-4b25-8720-1b6152616e6f req-e5dac4fd-da74-4768-951a-cb24a66d03b1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:38.937177 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-017d4798-09e9-4b25-8720-1b6152616e6f req-e5dac4fd-da74-4768-951a-cb24a66d03b1 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff&tenant_id=39febf983bf24778a300f287dbeca829 HTTP/1.1" status: 200 len: 1096 time: 0.0956600 May 20 04:29:38.940064 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-b8ddb67f-c8bc-48a3-833c-acdd0e70f6a2 tempest-TestMinimumBasicScenario-1108288795 tempest-TestMinimumBasicScenario-1108288795-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:38.941510 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-b8ddb67f-c8bc-48a3-833c-acdd0e70f6a2 tempest-TestMinimumBasicScenario-1108288795 tempest-TestMinimumBasicScenario-1108288795-project-admin] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 1094 time: 0.1812422 May 20 04:29:38.944520 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 41994) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:29:38.949816 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Request body: {'floatingip': {'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'port_id': '5f77cfe6-85e2-4a40-9a27-6ed37313ae13', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'fixed_ip_address': '172.20.0.12'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:29:38.956030 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:29:38.960025 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Usage tracker for resource:floatingip and tenant:13d0e12c764f483ea457246b10c4cf7e is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:29:38.963000 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:29:38.973277 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:29:39.027519 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for floatingip, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:39.130734 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-017d4798-09e9-4b25-8720-1b6152616e6f req-d3abc476-e7db-4f5c-a210-80ebc27f0b17 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:39.132285 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-017d4798-09e9-4b25-8720-1b6152616e6f req-d3abc476-e7db-4f5c-a210-80ebc27f0b17 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=39febf983bf24778a300f287dbeca829&device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 1096 time: 0.1602757 May 20 04:29:39.288129 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-017d4798-09e9-4b25-8720-1b6152616e6f req-86d3d2ea-a551-44c1-82fa-dbb8f93d38a9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:39.289524 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-017d4798-09e9-4b25-8720-1b6152616e6f req-86d3d2ea-a551-44c1-82fa-dbb8f93d38a9 service neutron] 10.222.0.74 "GET /v2.0/networks?id=e57f6e29-4637-455e-af9d-0975c45c995c HTTP/1.1" status: 200 len: 909 time: 0.1418786 May 20 04:29:39.290838 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:39.451438 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e8fa396b-d38f-45fd-8880-b1976ccae89a req-2d362f2c-9984-42de-92a0-ff027154de2d tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:39.453244 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e8fa396b-d38f-45fd-8880-b1976ccae89a req-2d362f2c-9984-42de-92a0-ff027154de2d tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.0874434 May 20 04:29:39.502413 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e8fa396b-d38f-45fd-8880-b1976ccae89a req-3a0afda9-5e40-48bb-96a0-dae19329197c tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0430970 May 20 04:29:39.556397 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-017d4798-09e9-4b25-8720-1b6152616e6f req-ab2995a2-82a4-47eb-98aa-356201643265 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=b858096a-44b8-4537-a167-0e9c03347445 HTTP/1.1" status: 200 len: 999 time: 0.2317722 May 20 04:29:39.673977 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Allocated IP 172.27.0.74 (712ce8c4-8feb-4719-bf13-e06992125cd9/e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5/6fc31c9c-218e-4616-bda1-b00cba81706e) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:29:39.676814 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-017d4798-09e9-4b25-8720-1b6152616e6f req-6f45e376-46a2-4946-94ba-b7149d309ba9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:39.678263 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-017d4798-09e9-4b25-8720-1b6152616e6f req-6f45e376-46a2-4946-94ba-b7149d309ba9 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=49523a89-260b-4732-abff-507d74fed4c4 HTTP/1.1" status: 200 len: 847 time: 0.0877764 May 20 04:29:39.877261 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:39.883408 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-017d4798-09e9-4b25-8720-1b6152616e6f req-1cf89b5a-fe33-482a-8fb2-f712f8e26a79 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:39.884907 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-017d4798-09e9-4b25-8720-1b6152616e6f req-1cf89b5a-fe33-482a-8fb2-f712f8e26a79 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=e57f6e29-4637-455e-af9d-0975c45c995c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1817729 May 20 04:29:39.996649 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:40.002286 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-9770e281-b7d1-4c65-8a54-43975627d9d4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:40.082518 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:29:40.088415 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:29:40.095877 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:29:40.122498 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Pushing event updated for resources: {'Port': ['ID=6fc31c9c-218e-4616-bda1-b00cba81706e,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:40.126333 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-017d4798-09e9-4b25-8720-1b6152616e6f req-176383b3-6299-4be8-a187-086d6fa8cf66 service neutron] 10.222.0.74 "GET /v2.0/networks/e57f6e29-4637-455e-af9d-0975c45c995c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2280252 May 20 04:29:40.157286 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:40.157566 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:40.157934 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:40.158285 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:40.158635 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:40.158970 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:40.159348 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:40.159696 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:40.160042 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:40.160382 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:40.160719 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:40.161061 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:40.180935 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.179s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:40.360769 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-017d4798-09e9-4b25-8720-1b6152616e6f req-ff93b91a-4dce-40de-bca8-270227a70a70 service neutron] 10.222.0.74 "GET /v2.0/networks/e57f6e29-4637-455e-af9d-0975c45c995c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2231061 May 20 04:29:40.632614 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for floatingip, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:40.693760 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f3848ea8-f1cf-4086-9f8b-8dee6b482f14 req-524be8dc-909c-4361-b39d-aeb4f27c5f65 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:40.695232 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f3848ea8-f1cf-4086-9f8b-8dee6b482f14 req-524be8dc-909c-4361-b39d-aeb4f27c5f65 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.0823550 May 20 04:29:40.709804 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:40.740736 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f3848ea8-f1cf-4086-9f8b-8dee6b482f14 req-17d4126d-bcdc-4287-a766-ed320550d072 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0398180 May 20 04:29:40.913568 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:40.921685 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:40.927451 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-4de0131d-047f-4c70-b3eb-19869ea5c866 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:41.125130 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Pushing event updated for resources: {'Port': ['ID=6fc31c9c-218e-4616-bda1-b00cba81706e,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:41.129114 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.202s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:41.136060 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-8003251'] for floatingip, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:41.309236 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.db.l3_db [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Floating IP 096511d5-89fe-4bdf-b446-22275adf2d5a associated. External IP: 172.27.0.74, port: 5f77cfe6-85e2-4a40-9a27-6ed37313ae13. May 20 04:29:41.681753 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Router 7b630b1e-4360-4f2e-9053-be7e631bfa3a has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91644) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:29:41.862933 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:29:41.873623 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-dc5d3dc0-e35d-4aca-a2c3-7144e2574577 None None] Sync routers for ids ['7b630b1e-4360-4f2e-9053-be7e631bfa3a'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:29:41.888068 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:41.888368 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-dc5d3dc0-e35d-4aca-a2c3-7144e2574577 None None] Routers {'7b630b1e-4360-4f2e-9053-be7e631bfa3a'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:29:41.889974 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-dc5d3dc0-e35d-4aca-a2c3-7144e2574577 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 9, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 9, "log_agent_heartbeats": false, "routers": 9},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:29:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['7b630b1e-4360-4f2e-9053-be7e631bfa3a'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:29:41.943517 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c205a551-e5d6-4b6d-8175-9e71179ece0f req-b598d89c-649e-480c-a401-5922604e338d tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:41.944856 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 996 time: 2.9992914 May 20 04:29:41.945809 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'dfc47d7d-eb29-4bb6-a680-660af78ffcf3', 'tag': '5f77cfe6-85e2-4a40-9a27-6ed37313ae13'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:29:41.946391 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c205a551-e5d6-4b6d-8175-9e71179ece0f req-b598d89c-649e-480c-a401-5922604e338d tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.0960736 May 20 04:29:41.950551 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 42010) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:29:41.960454 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-20109a30-cd87-452e-b5b2-de50670d420a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Request body: {'security_group': {'name': 'tempest-secgroup-smoke-79860973', 'description': 'tempest-secgroup-smoke-79860973 description', 'project_id': '13d0e12c764f483ea457246b10c4cf7e'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:29:41.966678 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-20109a30-cd87-452e-b5b2-de50670d420a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:29:41.978513 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-20109a30-cd87-452e-b5b2-de50670d420a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:29:41.994264 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-20109a30-cd87-452e-b5b2-de50670d420a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.002275 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c205a551-e5d6-4b6d-8175-9e71179ece0f req-62fb1f41-c7bc-48a5-9e33-5e37508cddd7 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0460887 May 20 04:29:42.012117 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'dfc47d7d-eb29-4bb6-a680-660af78ffcf3', 'tag': '5f77cfe6-85e2-4a40-9a27-6ed37313ae13', 'status': 'completed', 'code': 200} May 20 04:29:42.022393 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-20109a30-cd87-452e-b5b2-de50670d420a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:29:42.028061 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-20109a30-cd87-452e-b5b2-de50670d420a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Usage tracker for resource:security_group_rule and tenant:13d0e12c764f483ea457246b10c4cf7e is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:29:42.029485 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-20109a30-cd87-452e-b5b2-de50670d420a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:29:42.070957 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-20109a30-cd87-452e-b5b2-de50670d420a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:29:42.096188 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-dc5d3dc0-e35d-4aca-a2c3-7144e2574577 None None] Set router 7b630b1e-4360-4f2e-9053-be7e631bfa3a gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:29:42.145788 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0d91bd6-bbcb-4d0c-b551-b2e7a694b815 req-8faa8d33-84dc-4114-9159-b5c2fb85b7c9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:42.147252 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b0d91bd6-bbcb-4d0c-b551-b2e7a694b815 req-8faa8d33-84dc-4114-9159-b5c2fb85b7c9 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=13d0e12c764f483ea457246b10c4cf7e&device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 1099 time: 0.1147668 May 20 04:29:42.201032 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-20109a30-cd87-452e-b5b2-de50670d420a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.248928 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-20109a30-cd87-452e-b5b2-de50670d420a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.256508 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-598520ef-dd00-45b8-89ed-1511d26b75ac admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:42.318687 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-20109a30-cd87-452e-b5b2-de50670d420a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Pushing event updated for resources: {'SecurityGroup': ['ID=3a2163a0-97a4-4285-8fe2-96f2f61334ec,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:42.328688 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-20109a30-cd87-452e-b5b2-de50670d420a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.072s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:42.331997 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-20109a30-cd87-452e-b5b2-de50670d420a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.333170 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-20109a30-cd87-452e-b5b2-de50670d420a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1790 time: 0.3817177 May 20 04:29:42.336412 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 42016) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:29:42.337813 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0d91bd6-bbcb-4d0c-b551-b2e7a694b815 req-ab82126b-99ef-40bf-8627-56986ec893ed service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:42.339185 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b0d91bd6-bbcb-4d0c-b551-b2e7a694b815 req-ab82126b-99ef-40bf-8627-56986ec893ed service neutron] 10.222.0.74 "GET /v2.0/networks?id=9798d104-d4de-413f-b41d-fb9d7e8a818e HTTP/1.1" status: 200 len: 914 time: 0.1809373 May 20 04:29:42.342480 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Request body: {'security_group_rule': {'security_group_id': '3a2163a0-97a4-4285-8fe2-96f2f61334ec', 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:29:42.348785 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:29:42.352637 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Usage tracker for resource:security_group_rule and tenant:13d0e12c764f483ea457246b10c4cf7e is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:29:42.355504 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:29:42.362215 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:29:42.379031 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.442973 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.451761 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.453809 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-0a67906b-4a1e-4c6c-9b71-28bd8605a232 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:42.457931 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b0d91bd6-bbcb-4d0c-b551-b2e7a694b815 req-91e88176-96c0-423d-bf2a-68d764d32f8d service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=5f77cfe6-85e2-4a40-9a27-6ed37313ae13 HTTP/1.1" status: 200 len: 999 time: 0.1004679 May 20 04:29:42.465846 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-dc5d3dc0-e35d-4aca-a2c3-7144e2574577 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'name': 'tempest-TestMinimumBasicScenario-22498061-router', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158'}], 'enable_snat': True}, 'gw_port_id': '160f1898-b408-446e-8492-c4b331489205', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:24Z', 'updated_at': '2021-05-20T04:28:28Z', 'revision_number': 4, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'gw_port': {'id': '160f1898-b408-446e-8492-c4b331489205', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:79:e7:e5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 544, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:25Z', 'updated_at': '2021-05-20T04:28:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': '096511d5-89fe-4bdf-b446-22275adf2d5a', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'floating_ip_address': '172.27.0.74', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'port_id': '5f77cfe6-85e2-4a40-9a27-6ed37313ae13', 'fixed_ip_address': '172.20.0.12', 'status': 'DOWN', 'standard_attr_id': 571, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e', 'mac_address': 'fa:16:3e:55:e5:39', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'dfc47d7d-eb29-4bb6-a680-660af78ffcf3', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:29:40Z', 'updated_at': '2021-05-20T04:29:40Z', 'revision_number': 0, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'bce609e7-1912-4202-bc3c-9028a36c0adb', 'name': '', 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'mac_address': 'fa:16:3e:57:64:c1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_interface', 'standard_attr_id': 545, 'fixed_ips': [{'subnet_id': 'c5423164-3d24-4562-871f-ab03914420dc', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:27Z', 'updated_at': '2021-05-20T04:28:34Z', 'revision_number': 5, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'subnets': [{'id': 'c5423164-3d24-4562-871f-ab03914420dc', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:29:42.470940 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-dc5d3dc0-e35d-4aca-a2c3-7144e2574577 None None] Checking router: 7b630b1e-4360-4f2e-9053-be7e631bfa3a for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:29:42.478223 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=96d69f3e-34bf-4ab8-bb5f-a6744838ea90,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:42.479889 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-dc5d3dc0-e35d-4aca-a2c3-7144e2574577 None None] Routers {'7b630b1e-4360-4f2e-9053-be7e631bfa3a'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:29:42.480875 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-dc5d3dc0-e35d-4aca-a2c3-7144e2574577 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 9, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 9, "log_agent_heartbeats": false, "routers": 9},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:29:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['7b630b1e-4360-4f2e-9053-be7e631bfa3a'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:29:42.489357 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:42.489833 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:42.490207 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:42.490464 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:42.490818 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:42.491242 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:42.491590 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:42.491922 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:42.492302 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:42.492620 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:42.492982 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:42.493318 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:29:42.496411 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.500250 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.1600053 May 20 04:29:42.500250 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:42.500835 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 42020) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:29:42.507362 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-3cb305e1-882c-464b-b60e-864d8ad339dd tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Request body: {'security_group_rule': {'security_group_id': '3a2163a0-97a4-4285-8fe2-96f2f61334ec', 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'egress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:29:42.513156 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-3cb305e1-882c-464b-b60e-864d8ad339dd tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:29:42.517171 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-3cb305e1-882c-464b-b60e-864d8ad339dd tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Usage tracker for resource:security_group_rule and tenant:13d0e12c764f483ea457246b10c4cf7e is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:29:42.520251 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-3cb305e1-882c-464b-b60e-864d8ad339dd tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:29:42.526891 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-3cb305e1-882c-464b-b60e-864d8ad339dd tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:29:42.529626 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0d91bd6-bbcb-4d0c-b551-b2e7a694b815 req-72b261ab-2ece-46ef-a765-9d398ab242d5 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:42.531321 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b0d91bd6-bbcb-4d0c-b551-b2e7a694b815 req-72b261ab-2ece-46ef-a765-9d398ab242d5 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=c5423164-3d24-4562-871f-ab03914420dc HTTP/1.1" status: 200 len: 852 time: 0.0625935 May 20 04:29:42.545323 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3cb305e1-882c-464b-b60e-864d8ad339dd tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.616501 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3cb305e1-882c-464b-b60e-864d8ad339dd tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.621470 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3cb305e1-882c-464b-b60e-864d8ad339dd tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.625390 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-d1d128bd-78e4-43c0-937a-8aa0200c5bc2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:42.650154 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3cb305e1-882c-464b-b60e-864d8ad339dd tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=ea31c67c-05ed-4e94-a633-00152b2920bf,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:42.652954 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-3cb305e1-882c-464b-b60e-864d8ad339dd tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.029s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:42.662392 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3cb305e1-882c-464b-b60e-864d8ad339dd tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.665290 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-6383e307-ecf7-44ae-a98a-bdf8e5d2807a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.668842 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-3cb305e1-882c-464b-b60e-864d8ad339dd tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 763 time: 0.1656930 May 20 04:29:42.670091 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 42024) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:29:42.676516 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-753f27f7-6516-45ed-96d5-7c77e31e83b4 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Request body: {'security_group_rule': {'security_group_id': '3a2163a0-97a4-4285-8fe2-96f2f61334ec', 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'protocol': 'icmp', 'direction': 'ingress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:29:42.682802 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-dc5d3dc0-e35d-4aca-a2c3-7144e2574577 None None] Set router 7b630b1e-4360-4f2e-9053-be7e631bfa3a gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:29:42.683713 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-753f27f7-6516-45ed-96d5-7c77e31e83b4 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:29:42.687689 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-753f27f7-6516-45ed-96d5-7c77e31e83b4 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Usage tracker for resource:security_group_rule and tenant:13d0e12c764f483ea457246b10c4cf7e is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:29:42.691255 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-753f27f7-6516-45ed-96d5-7c77e31e83b4 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Quota usage for security_group_rule was recalculated. Used quota:8. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:29:42.697815 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-753f27f7-6516-45ed-96d5-7c77e31e83b4 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 8; quota limit: 100; headroom:92 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:29:42.716196 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-753f27f7-6516-45ed-96d5-7c77e31e83b4 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.718422 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0d91bd6-bbcb-4d0c-b551-b2e7a694b815 req-c80a9ed1-e712-43c0-9474-3e853eea3561 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:42.720044 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b0d91bd6-bbcb-4d0c-b551-b2e7a694b815 req-c80a9ed1-e712-43c0-9474-3e853eea3561 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=9798d104-d4de-413f-b41d-fb9d7e8a818e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1820052 May 20 04:29:42.782356 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-753f27f7-6516-45ed-96d5-7c77e31e83b4 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.787316 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-753f27f7-6516-45ed-96d5-7c77e31e83b4 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.789341 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:42.811502 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-753f27f7-6516-45ed-96d5-7c77e31e83b4 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=e4f87940-20e3-4581-9921-c7b4ffa82072,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:42.814015 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-753f27f7-6516-45ed-96d5-7c77e31e83b4 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.025s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:42.821969 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-753f27f7-6516-45ed-96d5-7c77e31e83b4 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.823410 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-753f27f7-6516-45ed-96d5-7c77e31e83b4 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.1524291 May 20 04:29:42.826153 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 42028) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:29:42.831365 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-374e49c2-fea2-49db-9740-cfe13a15f819 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Request body: {'security_group_rule': {'security_group_id': '3a2163a0-97a4-4285-8fe2-96f2f61334ec', 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'protocol': 'icmp', 'direction': 'egress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:29:42.837200 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-374e49c2-fea2-49db-9740-cfe13a15f819 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:29:42.841056 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-374e49c2-fea2-49db-9740-cfe13a15f819 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Usage tracker for resource:security_group_rule and tenant:13d0e12c764f483ea457246b10c4cf7e is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:29:42.844284 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-374e49c2-fea2-49db-9740-cfe13a15f819 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Quota usage for security_group_rule was recalculated. Used quota:9. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:29:42.850117 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-374e49c2-fea2-49db-9740-cfe13a15f819 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 9; quota limit: 100; headroom:91 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:29:42.867534 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-374e49c2-fea2-49db-9740-cfe13a15f819 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.868542 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b0d91bd6-bbcb-4d0c-b551-b2e7a694b815 req-ac9627b8-53f6-4193-aeb5-3c75fa2ed990 service neutron] 10.222.0.74 "GET /v2.0/networks/9798d104-d4de-413f-b41d-fb9d7e8a818e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1391788 May 20 04:29:42.932594 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-374e49c2-fea2-49db-9740-cfe13a15f819 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.936895 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-374e49c2-fea2-49db-9740-cfe13a15f819 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.943394 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9973322c-5078-4782-945b-3ed64a85ac24 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:42.981825 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-374e49c2-fea2-49db-9740-cfe13a15f819 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=150e121a-8677-49da-8f62-1dcdea95f8b2,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:42.985878 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-374e49c2-fea2-49db-9740-cfe13a15f819 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.042s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:42.988020 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-374e49c2-fea2-49db-9740-cfe13a15f819 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:42.990073 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-374e49c2-fea2-49db-9740-cfe13a15f819 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 768 time: 0.1629825 May 20 04:29:42.991966 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 42034) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:29:42.999799 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-78a9f2b4-f9dc-4592-b610-d542a6e1ed67 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Request body: {'security_group_rule': {'security_group_id': '3a2163a0-97a4-4285-8fe2-96f2f61334ec', 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'ingress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:29:43.005729 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-78a9f2b4-f9dc-4592-b610-d542a6e1ed67 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:29:43.010266 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-78a9f2b4-f9dc-4592-b610-d542a6e1ed67 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Usage tracker for resource:security_group_rule and tenant:13d0e12c764f483ea457246b10c4cf7e is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:29:43.013930 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-78a9f2b4-f9dc-4592-b610-d542a6e1ed67 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Quota usage for security_group_rule was recalculated. Used quota:10. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:29:43.020331 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-b0d91bd6-bbcb-4d0c-b551-b2e7a694b815 req-a87dc7ba-d7ec-43cc-af54-fd1682ac6103 service neutron] 10.222.0.74 "GET /v2.0/networks/9798d104-d4de-413f-b41d-fb9d7e8a818e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1406267 May 20 04:29:43.020841 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-78a9f2b4-f9dc-4592-b610-d542a6e1ed67 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 10; quota limit: 100; headroom:90 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:29:43.032599 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.securitygroups_db [None req-78a9f2b4-f9dc-4592-b610-d542a6e1ed67 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Project 13d0e12c764f483ea457246b10c4cf7e added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 20 04:29:43.039680 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-78a9f2b4-f9dc-4592-b610-d542a6e1ed67 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:43.048992 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-dc5d3dc0-e35d-4aca-a2c3-7144e2574577 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'name': 'tempest-TestMinimumBasicScenario-22498061-router', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158'}], 'enable_snat': True}, 'gw_port_id': '160f1898-b408-446e-8492-c4b331489205', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:24Z', 'updated_at': '2021-05-20T04:28:28Z', 'revision_number': 4, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'gw_port': {'id': '160f1898-b408-446e-8492-c4b331489205', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:79:e7:e5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 544, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:25Z', 'updated_at': '2021-05-20T04:28:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': '096511d5-89fe-4bdf-b446-22275adf2d5a', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'floating_ip_address': '172.27.0.74', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'port_id': '5f77cfe6-85e2-4a40-9a27-6ed37313ae13', 'fixed_ip_address': '172.20.0.12', 'status': 'DOWN', 'standard_attr_id': 571, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e', 'mac_address': 'fa:16:3e:55:e5:39', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'dfc47d7d-eb29-4bb6-a680-660af78ffcf3', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:29:40Z', 'updated_at': '2021-05-20T04:29:40Z', 'revision_number': 0, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'bce609e7-1912-4202-bc3c-9028a36c0adb', 'name': '', 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'mac_address': 'fa:16:3e:57:64:c1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_interface', 'standard_attr_id': 545, 'fixed_ips': [{'subnet_id': 'c5423164-3d24-4562-871f-ab03914420dc', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:27Z', 'updated_at': '2021-05-20T04:28:34Z', 'revision_number': 5, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'subnets': [{'id': 'c5423164-3d24-4562-871f-ab03914420dc', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:29:43.053631 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-dc5d3dc0-e35d-4aca-a2c3-7144e2574577 None None] The sync data for ids ['7b630b1e-4360-4f2e-9053-be7e631bfa3a'] in dv-cinder-782405-9-iscsi is: [{'id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'name': 'tempest-TestMinimumBasicScenario-22498061-router', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158'}], 'enable_snat': True}, 'gw_port_id': '160f1898-b408-446e-8492-c4b331489205', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:24Z', 'updated_at': '2021-05-20T04:28:28Z', 'revision_number': 4, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'gw_port': {'id': '160f1898-b408-446e-8492-c4b331489205', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:79:e7:e5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 544, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:25Z', 'updated_at': '2021-05-20T04:28:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': '096511d5-89fe-4bdf-b446-22275adf2d5a', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'floating_ip_address': '172.27.0.74', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'port_id': '5f77cfe6-85e2-4a40-9a27-6ed37313ae13', 'fixed_ip_address': '172.20.0.12', 'status': 'DOWN', 'standard_attr_id': 571, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e', 'mac_address': 'fa:16:3e:55:e5:39', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'dfc47d7d-eb29-4bb6-a680-660af78ffcf3', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:29:40Z', 'updated_at': '2021-05-20T04:29:40Z', 'revision_number': 0, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'bce609e7-1912-4202-bc3c-9028a36c0adb', 'name': '', 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'mac_address': 'fa:16:3e:57:64:c1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_interface', 'standard_attr_id': 545, 'fixed_ips': [{'subnet_id': 'c5423164-3d24-4562-871f-ab03914420dc', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:27Z', 'updated_at': '2021-05-20T04:28:34Z', 'revision_number': 5, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'subnets': [{'id': 'c5423164-3d24-4562-871f-ab03914420dc', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:29:43.127878 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-78a9f2b4-f9dc-4592-b610-d542a6e1ed67 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:43.133926 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-78a9f2b4-f9dc-4592-b610-d542a6e1ed67 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:43.136231 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-753f27f7-6516-45ed-96d5-7c77e31e83b4 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:43.158801 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-78a9f2b4-f9dc-4592-b610-d542a6e1ed67 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=a64e8cc6-a1b2-4306-b5ab-d24a40c5ba3b,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:43.196239 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-78a9f2b4-f9dc-4592-b610-d542a6e1ed67 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.060s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:43.214903 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-78a9f2b4-f9dc-4592-b610-d542a6e1ed67 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:43.216169 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-78a9f2b4-f9dc-4592-b610-d542a6e1ed67 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 774 time: 0.2232971 May 20 04:29:43.219577 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 42040) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:29:43.226236 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-a6ada475-0a92-443d-91e4-a46b85c4177c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Request body: {'security_group_rule': {'security_group_id': '3a2163a0-97a4-4285-8fe2-96f2f61334ec', 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'egress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:29:43.234049 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-a6ada475-0a92-443d-91e4-a46b85c4177c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:29:43.239451 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-a6ada475-0a92-443d-91e4-a46b85c4177c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Usage tracker for resource:security_group_rule and tenant:13d0e12c764f483ea457246b10c4cf7e is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:29:43.243572 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-a6ada475-0a92-443d-91e4-a46b85c4177c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Quota usage for security_group_rule was recalculated. Used quota:11. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:29:43.252937 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-a6ada475-0a92-443d-91e4-a46b85c4177c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 11; quota limit: 100; headroom:89 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:29:43.259584 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c5c178ae-ee6e-43dd-802d-fb8492984e1a None None] New status for floating IP 096511d5-89fe-4bdf-b446-22275adf2d5a: ACTIVE {{(pid=91646) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 20 04:29:43.266324 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.securitygroups_db [None req-a6ada475-0a92-443d-91e4-a46b85c4177c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Project 13d0e12c764f483ea457246b10c4cf7e added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 20 04:29:43.278611 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-06e26310-6f6a-4192-83ad-154b16b9febd req-ab3bf0de-3a32-46e1-8eb7-92b6f0fbc9e9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:43.280083 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-06e26310-6f6a-4192-83ad-154b16b9febd req-ab3bf0de-3a32-46e1-8eb7-92b6f0fbc9e9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.1273489 May 20 04:29:43.281342 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a6ada475-0a92-443d-91e4-a46b85c4177c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:43.360619 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-06e26310-6f6a-4192-83ad-154b16b9febd req-7f9d00b4-ed42-4183-a853-312c4179d117 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0722742 May 20 04:29:43.383310 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a6ada475-0a92-443d-91e4-a46b85c4177c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:43.388347 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a6ada475-0a92-443d-91e4-a46b85c4177c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:43.394556 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-ca3a57a4-0c6f-493b-b784-df38b85f1873 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:43.413623 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a6ada475-0a92-443d-91e4-a46b85c4177c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=093742b5-8ab6-4ec5-aefa-fc66aaa46713,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:43.416256 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-a6ada475-0a92-443d-91e4-a46b85c4177c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.022s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:43.424245 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a6ada475-0a92-443d-91e4-a46b85c4177c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:43.425432 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-a6ada475-0a92-443d-91e4-a46b85c4177c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 773 time: 0.2052619 May 20 04:29:43.549115 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d6cf940f-1cd6-4543-83cf-da9f65a0bad9 req-708376a8-680a-4583-8bba-8d0c0b38d4b7 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-79860973&fields=id&tenant_id=13d0e12c764f483ea457246b10c4cf7e HTTP/1.1" status: 200 len: 267 time: 0.0435331 May 20 04:29:43.639875 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d6cf940f-1cd6-4543-83cf-da9f65a0bad9 req-7fef1e0d-efa1-4560-af6d-6507aa5d31c8 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:43.641336 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d6cf940f-1cd6-4543-83cf-da9f65a0bad9 req-7fef1e0d-efa1-4560-af6d-6507aa5d31c8 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 946 time: 0.0865271 May 20 04:29:43.652154 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [req-d6cf940f-1cd6-4543-83cf-da9f65a0bad9 req-56ed50e2-ae99-461d-b0bd-5a15bd728de6 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Request body: {'port': {'security_groups': ['f596c737-91de-4c7f-9aa1-e6e95d668a70', '3a2163a0-97a4-4285-8fe2-96f2f61334ec']}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:29:43.773195 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-d6cf940f-1cd6-4543-83cf-da9f65a0bad9 req-56ed50e2-ae99-461d-b0bd-5a15bd728de6 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:44.112433 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-d6cf940f-1cd6-4543-83cf-da9f65a0bad9 req-56ed50e2-ae99-461d-b0bd-5a15bd728de6 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] For port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=5f77cfe6-85e2-4a40-9a27-6ed37313ae13,segment=NetworkSegment(d558b057-1c72-4efe-ab53-567b572e2de5),segment_id=d558b057-1c72-4efe-ab53-567b572e2de5)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:29:44.139716 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-d6cf940f-1cd6-4543-83cf-da9f65a0bad9 req-56ed50e2-ae99-461d-b0bd-5a15bd728de6 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for port, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:44.147236 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-d6cf940f-1cd6-4543-83cf-da9f65a0bad9 req-56ed50e2-ae99-461d-b0bd-5a15bd728de6 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:44.153017 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-86fe4049-72c5-4a6f-9bc0-30bd2acce14f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:44.160565 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['5f77cfe6-85e2-4a40-9a27-6ed37313ae13'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:29:44.217223 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d6cf940f-1cd6-4543-83cf-da9f65a0bad9 req-56ed50e2-ae99-461d-b0bd-5a15bd728de6 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:44.217849 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-d6cf940f-1cd6-4543-83cf-da9f65a0bad9 req-56ed50e2-ae99-461d-b0bd-5a15bd728de6 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:44.219235 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d6cf940f-1cd6-4543-83cf-da9f65a0bad9 req-56ed50e2-ae99-461d-b0bd-5a15bd728de6 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "PUT /v2.0/ports/5f77cfe6-85e2-4a40-9a27-6ed37313ae13 HTTP/1.1" status: 200 len: 982 time: 0.5715692 May 20 04:29:44.219757 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'dfc47d7d-eb29-4bb6-a680-660af78ffcf3', 'tag': '5f77cfe6-85e2-4a40-9a27-6ed37313ae13'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:29:44.240490 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-56ed50e2-ae99-461d-b0bd-5a15bd728de6 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Pushing event updated for resources: {'Port': ['ID=5f77cfe6-85e2-4a40-9a27-6ed37313ae13,revision_number=6']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:44.244045 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-56ed50e2-ae99-461d-b0bd-5a15bd728de6 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.091s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:44.270182 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'dfc47d7d-eb29-4bb6-a680-660af78ffcf3', 'tag': '5f77cfe6-85e2-4a40-9a27-6ed37313ae13', 'status': 'completed', 'code': 200} May 20 04:29:44.376973 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7f0c3afb-84cf-4bb0-b38c-562703fbdaa6 req-ee093486-2a46-43d2-ac3f-de5a878ddef5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:44.378602 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7f0c3afb-84cf-4bb0-b38c-562703fbdaa6 req-ee093486-2a46-43d2-ac3f-de5a878ddef5 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=13d0e12c764f483ea457246b10c4cf7e&device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 1138 time: 0.0858765 May 20 04:29:44.381102 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['5f77cfe6-85e2-4a40-9a27-6ed37313ae13'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:29:44.490424 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c5c8329d-07ed-484b-869b-497828d98a53 req-c1b93945-3872-4c6b-b6fd-6306339d27d1 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:44.492122 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c5c8329d-07ed-484b-869b-497828d98a53 req-c1b93945-3872-4c6b-b6fd-6306339d27d1 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.1237776 May 20 04:29:44.527967 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7f0c3afb-84cf-4bb0-b38c-562703fbdaa6 req-6c5b301a-e9f0-4141-a029-331ec96891f0 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:44.529349 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7f0c3afb-84cf-4bb0-b38c-562703fbdaa6 req-6c5b301a-e9f0-4141-a029-331ec96891f0 service neutron] 10.222.0.74 "GET /v2.0/networks?id=9798d104-d4de-413f-b41d-fb9d7e8a818e HTTP/1.1" status: 200 len: 914 time: 0.1422856 May 20 04:29:44.554173 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-05c46286-201a-4d27-9b3c-21debc2493a1 None None] Provisioning complete for port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:29:44.554729 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-05c46286-201a-4d27-9b3c-21debc2493a1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:44.593329 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c5c8329d-07ed-484b-869b-497828d98a53 req-bdabf453-971b-491c-8bd0-73d28e54e095 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0660737 May 20 04:29:44.619697 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-50b302b7-77a4-4961-b10d-07b2b81d8d10 req-f0a7b19b-3ee1-44b9-9278-8f572a3447b5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:44.626830 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-50b302b7-77a4-4961-b10d-07b2b81d8d10 req-f0a7b19b-3ee1-44b9-9278-8f572a3447b5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.1299760 May 20 04:29:44.641041 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7f0c3afb-84cf-4bb0-b38c-562703fbdaa6 req-bf0e7e0a-da9f-4d84-be22-d37247a08683 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=5f77cfe6-85e2-4a40-9a27-6ed37313ae13 HTTP/1.1" status: 200 len: 1001 time: 0.1048729 May 20 04:29:44.668086 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-50b302b7-77a4-4961-b10d-07b2b81d8d10 req-d1ef0813-4ebe-4c31-810d-bc23f4472af8 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0403872 May 20 04:29:44.707342 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7f0c3afb-84cf-4bb0-b38c-562703fbdaa6 req-fd8b1fbd-73cf-4b54-8bef-608a2909bc43 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:44.708766 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7f0c3afb-84cf-4bb0-b38c-562703fbdaa6 req-fd8b1fbd-73cf-4b54-8bef-608a2909bc43 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=c5423164-3d24-4562-871f-ab03914420dc HTTP/1.1" status: 200 len: 852 time: 0.0614517 May 20 04:29:44.784536 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7f0c3afb-84cf-4bb0-b38c-562703fbdaa6 req-9bd498a0-077f-42b4-8ff0-16c8ce4c2c40 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:44.786033 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7f0c3afb-84cf-4bb0-b38c-562703fbdaa6 req-9bd498a0-077f-42b4-8ff0-16c8ce4c2c40 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=9798d104-d4de-413f-b41d-fb9d7e8a818e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0698032 May 20 04:29:44.919690 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7f0c3afb-84cf-4bb0-b38c-562703fbdaa6 req-4d46d591-0fe3-492d-b4b1-9f075daa92fd service neutron] 10.222.0.74 "GET /v2.0/networks/9798d104-d4de-413f-b41d-fb9d7e8a818e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1240819 May 20 04:29:45.005437 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:45.052963 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7f0c3afb-84cf-4bb0-b38c-562703fbdaa6 req-51c1bc55-1302-42c1-ad48-12b65de59af5 service neutron] 10.222.0.74 "GET /v2.0/networks/9798d104-d4de-413f-b41d-fb9d7e8a818e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1250770 May 20 04:29:45.106841 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-3d6967cb-95f1-43eb-a97d-8a0040232a96 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:45.862933 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bd50fbf5-d9b9-404d-9046-0ac24327d10f req-c2f5310e-eb0e-4f6e-b9a6-83718f8d077d tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:45.864400 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bd50fbf5-d9b9-404d-9046-0ac24327d10f req-c2f5310e-eb0e-4f6e-b9a6-83718f8d077d tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.0893164 May 20 04:29:45.915721 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bd50fbf5-d9b9-404d-9046-0ac24327d10f req-edf5807b-5f4b-4204-a276-8982152be1e2 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0426366 May 20 04:29:46.206295 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:47.132201 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-efc1d24c-73ab-45d0-8eba-e87cd2a5d49e req-690f3adc-f496-4324-b44d-4d8a1b61745d tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:47.133709 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-efc1d24c-73ab-45d0-8eba-e87cd2a5d49e req-690f3adc-f496-4324-b44d-4d8a1b61745d tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.1025929 May 20 04:29:47.182284 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-efc1d24c-73ab-45d0-8eba-e87cd2a5d49e req-63e0fb00-2cf7-4857-a3f9-2efa3e8e388b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0396907 May 20 04:29:47.851181 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:48.401785 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75b9056c-0fdf-4152-b800-d7cc3d363677 req-42e1dae9-dbfc-4b0c-9885-b8b2e48342e5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:48.403342 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-75b9056c-0fdf-4152-b800-d7cc3d363677 req-42e1dae9-dbfc-4b0c-9885-b8b2e48342e5 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.0926619 May 20 04:29:48.450383 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-75b9056c-0fdf-4152-b800-d7cc3d363677 req-02aaab69-4929-4a45-b90c-7a4af86b1596 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0412669 May 20 04:29:49.659615 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1f7598be-9d72-453a-848b-4c8b8f094958 req-1db1ddce-5022-4a42-b5d5-8f5d69ca35b6 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:49.661055 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1f7598be-9d72-453a-848b-4c8b8f094958 req-1db1ddce-5022-4a42-b5d5-8f5d69ca35b6 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 943 time: 0.0895143 May 20 04:29:49.710188 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1f7598be-9d72-453a-848b-4c8b8f094958 req-d02a9ded-978c-454e-a34c-05a63ceffdef tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0432298 May 20 04:29:50.078148 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b858096a-44b8-4537-a167-0e9c03347445 details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:29:50.221913 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b858096a-44b8-4537-a167-0e9c03347445, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b858096a-44b8-4537-a167-0e9c03347445,segment=NetworkSegment(c7d77442-5ac6-48cf-ab59-3f5962fc1e3e),segment_id=c7d77442-5ac6-48cf-ab59-3f5962fc1e3e)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:29:50.236081 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': 'b858096a-44b8-4537-a167-0e9c03347445', 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c', 'port_id': 'b858096a-44b8-4537-a167-0e9c03347445', 'mac_address': 'fa:16:3e:6c:52:6d', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 806, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '49523a89-260b-4732-abff-507d74fed4c4', 'ip_address': '172.20.0.5'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:29:50.299128 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:50.325440 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id b858096a-44b8-4537-a167-0e9c03347445 {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:29:50.342315 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b858096a-44b8-4537-a167-0e9c03347445, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b858096a-44b8-4537-a167-0e9c03347445,segment=NetworkSegment(c7d77442-5ac6-48cf-ab59-3f5962fc1e3e),segment_id=c7d77442-5ac6-48cf-ab59-3f5962fc1e3e)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:29:50.348054 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:50.349797 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:50.416724 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=b858096a-44b8-4537-a167-0e9c03347445,revision_number=7']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:50.419950 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.070s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:50.773090 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['b858096a-44b8-4537-a167-0e9c03347445'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:29:50.920800 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6366c7dd-7269-44d4-925f-49fbc8876e9d req-401ff153-ba20-4a35-9793-4dd761761cfd tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:50.922273 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6366c7dd-7269-44d4-925f-49fbc8876e9d req-401ff153-ba20-4a35-9793-4dd761761cfd tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 944 time: 0.0857835 May 20 04:29:50.967129 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6366c7dd-7269-44d4-925f-49fbc8876e9d req-87959779-43b6-4d73-a4fb-ff047d1e426b tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0381830 May 20 04:29:52.092123 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b858096a-44b8-4537-a167-0e9c03347445 up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:29:52.168764 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port b858096a-44b8-4537-a167-0e9c03347445 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:29:52.169405 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:52.169745 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e1615ae9-63c0-4146-80c5-53cb531af50e req-6b321810-a8f5-4b14-a727-d38531aca313 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:52.171482 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e1615ae9-63c0-4146-80c5-53cb531af50e req-6b321810-a8f5-4b14-a727-d38531aca313 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 944 time: 0.0891244 May 20 04:29:52.220973 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e1615ae9-63c0-4146-80c5-53cb531af50e req-54c64003-106f-4c83-89a5-335e820642de tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0434752 May 20 04:29:52.285030 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:52.312225 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '30d5e70e-9acd-400c-83be-01301f56a5ff', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b858096a-44b8-4537-a167-0e9c03347445'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:29:52.322035 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b858096a-44b8-4537-a167-0e9c03347445, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b858096a-44b8-4537-a167-0e9c03347445,segment=NetworkSegment(c7d77442-5ac6-48cf-ab59-3f5962fc1e3e),segment_id=c7d77442-5ac6-48cf-ab59-3f5962fc1e3e)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:29:52.327301 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:29:52.329222 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-396589ca-e8a5-4b2d-8109-78f4c5e544d1 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:29:52.364206 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '30d5e70e-9acd-400c-83be-01301f56a5ff', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b858096a-44b8-4537-a167-0e9c03347445', 'code': 200} May 20 04:29:52.395946 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=b858096a-44b8-4537-a167-0e9c03347445,revision_number=8']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:29:52.398923 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.070s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:29:52.668378 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5a6c49e9-3de2-4318-b8eb-bc4f3b9a3e8e req-7ff339d9-a2b1-4ebc-966b-ea2bcfa79e12 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:52.669905 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-5a6c49e9-3de2-4318-b8eb-bc4f3b9a3e8e req-7ff339d9-a2b1-4ebc-966b-ea2bcfa79e12 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 1098 time: 0.0745723 May 20 04:29:52.716545 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-5a6c49e9-3de2-4318-b8eb-bc4f3b9a3e8e req-cf594640-7227-4ebc-b3c1-ee3fa2c156c6 service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0393286 May 20 04:29:53.421985 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-78feaa7e-0634-4e06-88b4-047639d5eca7 req-3a6f6849-bb73-4c28-b39e-9fa47de19245 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:53.423379 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-78feaa7e-0634-4e06-88b4-047639d5eca7 req-3a6f6849-bb73-4c28-b39e-9fa47de19245 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.0887492 May 20 04:29:53.475353 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-78feaa7e-0634-4e06-88b4-047639d5eca7 req-a445355a-c0b4-4db4-8075-4045238ababe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0433943 May 20 04:29:54.678214 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3beed8ce-6f13-42fb-95ed-7178c36078a6 req-abd5bf9f-22af-436d-bf52-c2e93b165f3f tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:29:54.679651 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3beed8ce-6f13-42fb-95ed-7178c36078a6 req-abd5bf9f-22af-436d-bf52-c2e93b165f3f tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.0841672 May 20 04:29:54.725511 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3beed8ce-6f13-42fb-95ed-7178c36078a6 req-a8d90aaa-0e79-4214-834d-d09029a51d6f tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0399299 May 20 04:30:04.003026 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:30:05.629399 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:10.164034 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-1d59e014-68c1-4594-bf3e-58b3f216543d req-f45ad48c-07ac-4025-8622-0986dd738345 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0709476 May 20 04:30:10.268061 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d59e014-68c1-4594-bf3e-58b3f216543d req-c98585a9-f48a-4df8-9784-705bc759b87a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:10.269397 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-1d59e014-68c1-4594-bf3e-58b3f216543d req-c98585a9-f48a-4df8-9784-705bc759b87a service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=39febf983bf24778a300f287dbeca829&device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 1098 time: 0.0816200 May 20 04:30:10.421302 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d59e014-68c1-4594-bf3e-58b3f216543d req-65096cac-f856-4f89-8716-19d86f7a05c5 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:10.422755 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-1d59e014-68c1-4594-bf3e-58b3f216543d req-65096cac-f856-4f89-8716-19d86f7a05c5 service neutron] 10.222.0.74 "GET /v2.0/networks?id=e57f6e29-4637-455e-af9d-0975c45c995c HTTP/1.1" status: 200 len: 909 time: 0.1331127 May 20 04:30:10.536284 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-1d59e014-68c1-4594-bf3e-58b3f216543d req-33127537-4894-4555-9f39-44fa182f95b9 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=b858096a-44b8-4537-a167-0e9c03347445 HTTP/1.1" status: 200 len: 1001 time: 0.0963590 May 20 04:30:10.702117 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d59e014-68c1-4594-bf3e-58b3f216543d req-7767cfd7-d539-40bd-9363-5a997ebfae75 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:10.704272 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-1d59e014-68c1-4594-bf3e-58b3f216543d req-7767cfd7-d539-40bd-9363-5a997ebfae75 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=49523a89-260b-4732-abff-507d74fed4c4 HTTP/1.1" status: 200 len: 847 time: 0.1522210 May 20 04:30:10.790912 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d59e014-68c1-4594-bf3e-58b3f216543d req-18c2312f-9dcd-4fa8-80b1-d6043e95a804 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:10.792273 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-1d59e014-68c1-4594-bf3e-58b3f216543d req-18c2312f-9dcd-4fa8-80b1-d6043e95a804 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=e57f6e29-4637-455e-af9d-0975c45c995c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0805018 May 20 04:30:10.935041 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-1d59e014-68c1-4594-bf3e-58b3f216543d req-5f3dd8e9-07fa-4b6b-954e-2a652d79e973 service neutron] 10.222.0.74 "GET /v2.0/networks/e57f6e29-4637-455e-af9d-0975c45c995c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1352935 May 20 04:30:11.072855 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-1d59e014-68c1-4594-bf3e-58b3f216543d req-54e1409d-3245-4fb5-95dd-2c0f60b9c1da service neutron] 10.222.0.74 "GET /v2.0/networks/e57f6e29-4637-455e-af9d-0975c45c995c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1293647 May 20 04:30:12.622301 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-05108ab8-1637-4d99-be5f-38c5861ee03d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:14.979747 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:15.090906 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-4e24dbe6-1e79-4658-9ab7-ef6b7981eb43 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:16.214691 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:17.852297 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:35.632623 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:41.008339 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:30:41.408194 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-48580b0e-5b81-4b64-ad1f-d4a6d36a3c92 req-83e47baf-fa83-4bbe-89da-4bee6a2c1d6c tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:41.409831 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-48580b0e-5b81-4b64-ad1f-d4a6d36a3c92 req-83e47baf-fa83-4bbe-89da-4bee6a2c1d6c tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0814054 May 20 04:30:41.455049 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-48580b0e-5b81-4b64-ad1f-d4a6d36a3c92 req-2fbc0667-fcb3-42c9-b34b-5333b7454b9e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0394330 May 20 04:30:42.617802 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a9f769f-ae20-4b14-959a-68227ad25550 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:42.652880 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-000dcabe-f371-4d1d-926e-5a505c6a4c6e req-a1c62426-bce0-40f6-8710-089f030477e8 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:42.654341 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-000dcabe-f371-4d1d-926e-5a505c6a4c6e req-a1c62426-bce0-40f6-8710-089f030477e8 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.0906136 May 20 04:30:42.700571 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-000dcabe-f371-4d1d-926e-5a505c6a4c6e req-322f4230-def9-485d-b5a6-622949dcf1ca tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=a510b283-ab41-46f8-afd9-15c93cb76abc&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0400708 May 20 04:30:42.832533 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-41359ccb-b509-4992-940a-5d41d626bd67 req-9b57d5a3-ef59-413b-9988-7087b96b55b5 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:42.833218 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-41359ccb-b509-4992-940a-5d41d626bd67 req-9b57d5a3-ef59-413b-9988-7087b96b55b5 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Synchronizing usage tracker for tenant:198bc8ab083f4a729f32b8ef27a6ee77 on resource:port {{(pid=91644) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:30:42.848769 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-41359ccb-b509-4992-940a-5d41d626bd67 req-9b57d5a3-ef59-413b-9988-7087b96b55b5 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Unset dirty status for tenant:198bc8ab083f4a729f32b8ef27a6ee77 on resource:port {{(pid=91644) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:30:42.850029 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-41359ccb-b509-4992-940a-5d41d626bd67 req-9b57d5a3-ef59-413b-9988-7087b96b55b5 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.74 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 945 time: 0.1001468 May 20 04:30:42.986782 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [req-41359ccb-b509-4992-940a-5d41d626bd67 req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Deleting port 2cf59c2c-723c-4450-8b98-0809b16290a8 {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:30:42.987130 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-41359ccb-b509-4992-940a-5d41d626bd67 req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:43.120220 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-41359ccb-b509-4992-940a-5d41d626bd67 req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] For port 2cf59c2c-723c-4450-8b98-0809b16290a8, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=2cf59c2c-723c-4450-8b98-0809b16290a8,segment=NetworkSegment(b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7),segment_id=b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:30:43.120497 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-41359ccb-b509-4992-940a-5d41d626bd67 req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:43.407388 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-41359ccb-b509-4992-940a-5d41d626bd67 req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for floatingip, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:43.407672 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-41359ccb-b509-4992-940a-5d41d626bd67 req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-8003251'] for floatingip, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:43.407975 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.db.l3_db [req-41359ccb-b509-4992-940a-5d41d626bd67 req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Floating IP b1bce8cf-eac1-4b98-800e-1ec838d5f270 disassociated. External IP: 172.27.0.159, port: 2cf59c2c-723c-4450-8b98-0809b16290a8. May 20 04:30:43.408332 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [req-41359ccb-b509-4992-940a-5d41d626bd67 req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Calling delete_port for 2cf59c2c-723c-4450-8b98-0809b16290a8 owned by compute:nova {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:30:43.517858 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-41359ccb-b509-4992-940a-5d41d626bd67 req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:43.523379 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-90a69dde-eeb6-4351-9908-3004a6c77f53 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:30:43.568414 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Pushing event deleted for resources: {'Port': ['ID=2cf59c2c-723c-4450-8b98-0809b16290a8,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:30:43.579574 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:43.579942 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:43.580389 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:43.580662 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:43.581019 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:43.581405 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:43.581774 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:43.582122 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:43.582470 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:43.582833 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:43.583229 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:43.583612 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:43.587157 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.064s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:30:43.767604 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [req-41359ccb-b509-4992-940a-5d41d626bd67 req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Router 10aeccef-e762-4811-a411-a1c9817db442 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91644) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:30:43.852951 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ae0ebf4e-7e8a-400d-95d6-7a4aaa3a0c8b req-e06ae51a-6b7b-4527-b502-ab8e43e1ab42 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b HTTP/1.1" status: 200 len: 210 time: 0.0427263 May 20 04:30:44.051132 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-41359ccb-b509-4992-940a-5d41d626bd67 req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:30:44.061205 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ede8b791-bc84-4d49-9ba4-e493697634a2 None None] Sync routers for ids ['10aeccef-e762-4811-a411-a1c9817db442'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:30:44.074471 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ede8b791-bc84-4d49-9ba4-e493697634a2 None None] Routers {'10aeccef-e762-4811-a411-a1c9817db442'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:30:44.075161 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-ede8b791-bc84-4d49-9ba4-e493697634a2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 9, "extensions": [], "floating_ips": 5, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 9, "log_agent_heartbeats": false, "routers": 9},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:30:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['10aeccef-e762-4811-a411-a1c9817db442'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:30:44.125716 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-41359ccb-b509-4992-940a-5d41d626bd67 req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] No DVR routers for this DVR port 2cf59c2c-723c-4450-8b98-0809b16290a8 on host hv-cinder-78240 {{(pid=91644) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:30:44.129125 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-41359ccb-b509-4992-940a-5d41d626bd67 req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:44.131763 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b', 'name': 'network-vif-deleted', 'tag': '2cf59c2c-723c-4450-8b98-0809b16290a8'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:30:44.148633 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-41359ccb-b509-4992-940a-5d41d626bd67 req-20eef99a-774f-423d-a2c9-0ac615cac4ea tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.74 "DELETE /v2.0/ports/2cf59c2c-723c-4450-8b98-0809b16290a8 HTTP/1.1" status: 204 len: 173 time: 1.2915831 May 20 04:30:44.181034 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '9953398b-ac14-4f0f-9a0e-9eb73c3c8e6b', 'name': 'network-vif-deleted', 'tag': '2cf59c2c-723c-4450-8b98-0809b16290a8', 'status': 'completed', 'code': 200} May 20 04:30:44.254955 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-ede8b791-bc84-4d49-9ba4-e493697634a2 None None] Set router 10aeccef-e762-4811-a411-a1c9817db442 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:30:44.604518 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-ede8b791-bc84-4d49-9ba4-e493697634a2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33'}], 'enable_snat': True}, 'gw_port_id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:24:36Z', 'revision_number': 4, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port': {'id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:64:1f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_gateway', 'standard_attr_id': 295, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:31Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '7dc69379-dd61-4add-8e0a-4ef00a211a0a', 'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'mac_address': 'fa:16:3e:a4:49:d4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_interface', 'standard_attr_id': 307, 'fixed_ips': [{'subnet_id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:35Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 5, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'subnets': [{'id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:30:44.608366 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ede8b791-bc84-4d49-9ba4-e493697634a2 None None] Checking router: 10aeccef-e762-4811-a411-a1c9817db442 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:30:44.616883 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ede8b791-bc84-4d49-9ba4-e493697634a2 None None] Routers {'10aeccef-e762-4811-a411-a1c9817db442'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:30:44.617492 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-ede8b791-bc84-4d49-9ba4-e493697634a2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 9, "extensions": [], "floating_ips": 5, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 9, "log_agent_heartbeats": false, "routers": 9},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:30:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['10aeccef-e762-4811-a411-a1c9817db442'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:30:44.806666 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-ede8b791-bc84-4d49-9ba4-e493697634a2 None None] Set router 10aeccef-e762-4811-a411-a1c9817db442 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:30:44.978243 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 42256) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:30:45.032989 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:45.046057 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-cfcc7efb-17fb-4212-b7ae-fdfe522afa52 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:45.074321 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-cfcc7efb-17fb-4212-b7ae-fdfe522afa52 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:45.120937 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-100cfe50-7f59-4ea9-a98b-ed5db886d9e0 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:45.133155 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-cfcc7efb-17fb-4212-b7ae-fdfe522afa52 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:45.137121 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-61797c5b-1dcb-46a3-a935-4eedd57ed2c9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:30:45.145539 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-ede8b791-bc84-4d49-9ba4-e493697634a2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33'}], 'enable_snat': True}, 'gw_port_id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:24:36Z', 'revision_number': 4, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port': {'id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:64:1f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_gateway', 'standard_attr_id': 295, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:31Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '7dc69379-dd61-4add-8e0a-4ef00a211a0a', 'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'mac_address': 'fa:16:3e:a4:49:d4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_interface', 'standard_attr_id': 307, 'fixed_ips': [{'subnet_id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:35Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 5, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'subnets': [{'id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:30:45.149067 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cfcc7efb-17fb-4212-b7ae-fdfe522afa52 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=a510b283-ab41-46f8-afd9-15c93cb76abc,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:30:45.149246 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ede8b791-bc84-4d49-9ba4-e493697634a2 None None] The sync data for ids ['10aeccef-e762-4811-a411-a1c9817db442'] in dv-cinder-782405-9-iscsi is: [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33'}], 'enable_snat': True}, 'gw_port_id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:24:36Z', 'revision_number': 4, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port': {'id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:64:1f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_gateway', 'standard_attr_id': 295, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:31Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '7dc69379-dd61-4add-8e0a-4ef00a211a0a', 'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'mac_address': 'fa:16:3e:a4:49:d4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_interface', 'standard_attr_id': 307, 'fixed_ips': [{'subnet_id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:35Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 5, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'subnets': [{'id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:30:45.159793 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cfcc7efb-17fb-4212-b7ae-fdfe522afa52 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:45.160147 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cfcc7efb-17fb-4212-b7ae-fdfe522afa52 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:45.160543 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cfcc7efb-17fb-4212-b7ae-fdfe522afa52 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:45.160861 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cfcc7efb-17fb-4212-b7ae-fdfe522afa52 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:45.161239 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cfcc7efb-17fb-4212-b7ae-fdfe522afa52 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:45.161558 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cfcc7efb-17fb-4212-b7ae-fdfe522afa52 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:45.161902 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cfcc7efb-17fb-4212-b7ae-fdfe522afa52 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:45.162247 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cfcc7efb-17fb-4212-b7ae-fdfe522afa52 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:45.162606 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cfcc7efb-17fb-4212-b7ae-fdfe522afa52 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:45.162936 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cfcc7efb-17fb-4212-b7ae-fdfe522afa52 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:45.163302 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cfcc7efb-17fb-4212-b7ae-fdfe522afa52 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:45.163619 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cfcc7efb-17fb-4212-b7ae-fdfe522afa52 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:45.166922 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-cfcc7efb-17fb-4212-b7ae-fdfe522afa52 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.030s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:30:45.181006 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-cfcc7efb-17fb-4212-b7ae-fdfe522afa52 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:45.182957 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-cfcc7efb-17fb-4212-b7ae-fdfe522afa52 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "DELETE /v2.0/security-groups/a510b283-ab41-46f8-afd9-15c93cb76abc HTTP/1.1" status: 204 len: 168 time: 0.2029312 May 20 04:30:45.185108 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 42262) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:30:45.202947 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.api.v2.resource [None req-9e376944-cde4-4e72-806b-de506a81301e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] show failed (client error): The resource could not be found. May 20 04:30:45.204160 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-9e376944-cde4-4e72-806b-de506a81301e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups/a510b283-ab41-46f8-afd9-15c93cb76abc HTTP/1.1" status: 404 len: 347 time: 0.0179017 May 20 04:30:45.207001 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 42266) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:30:45.374385 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-99ebdd83-8259-4ec0-92c5-7e3758680a56 None None] New status for floating IP b1bce8cf-eac1-4b98-800e-1ec838d5f270: DOWN {{(pid=91646) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 20 04:30:45.518576 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-40e71bf7-eda7-4d8d-82cc-fd490988a2ec tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Deleting port 4af94d38-0efa-4427-9792-914bf823abf2 {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:30:45.518904 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-40e71bf7-eda7-4d8d-82cc-fd490988a2ec tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:45.636120 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-40e71bf7-eda7-4d8d-82cc-fd490988a2ec tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:45.636465 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-40e71bf7-eda7-4d8d-82cc-fd490988a2ec tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:45.755257 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-40e71bf7-eda7-4d8d-82cc-fd490988a2ec tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Calling delete_port for 4af94d38-0efa-4427-9792-914bf823abf2 owned by network:floatingip {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:30:45.877872 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-40e71bf7-eda7-4d8d-82cc-fd490988a2ec tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:45.881327 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-8d107e4f-1057-4305-b09e-179c4265f0d7 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:30:45.923031 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-40e71bf7-eda7-4d8d-82cc-fd490988a2ec tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Pushing event deleted for resources: {'Port': ['ID=4af94d38-0efa-4427-9792-914bf823abf2,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:30:45.926080 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-40e71bf7-eda7-4d8d-82cc-fd490988a2ec tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:30:46.017525 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-40e71bf7-eda7-4d8d-82cc-fd490988a2ec tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for floatingip, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:46.032319 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-40e71bf7-eda7-4d8d-82cc-fd490988a2ec tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:46.033423 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-40e71bf7-eda7-4d8d-82cc-fd490988a2ec tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "DELETE /v2.0/floatingips/b1bce8cf-eac1-4b98-800e-1ec838d5f270 HTTP/1.1" status: 204 len: 168 time: 0.8254023 May 20 04:30:46.208286 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:46.325584 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 42276) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:30:46.330919 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-44cd32a5-bf0c-4840-9608-5c50289a01f7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Request body: {'security_group': {'name': 'tempest-securitygroup--439509829', 'description': 'tempest-description--317642544'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:30:46.336909 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-44cd32a5-bf0c-4840-9608-5c50289a01f7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:30:46.341165 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-44cd32a5-bf0c-4840-9608-5c50289a01f7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Usage tracker for resource:security_group and tenant:198bc8ab083f4a729f32b8ef27a6ee77 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:30:46.344868 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-44cd32a5-bf0c-4840-9608-5c50289a01f7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:30:46.350856 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-44cd32a5-bf0c-4840-9608-5c50289a01f7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:30:46.363092 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-44cd32a5-bf0c-4840-9608-5c50289a01f7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:46.386218 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-44cd32a5-bf0c-4840-9608-5c50289a01f7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:30:46.390365 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-44cd32a5-bf0c-4840-9608-5c50289a01f7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Usage tracker for resource:security_group_rule and tenant:198bc8ab083f4a729f32b8ef27a6ee77 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:30:46.393278 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-44cd32a5-bf0c-4840-9608-5c50289a01f7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:30:46.399044 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-44cd32a5-bf0c-4840-9608-5c50289a01f7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:30:46.423576 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 2cf59c2c-723c-4450-8b98-0809b16290a8 no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:30:46.458699 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: 2cf59c2c-723c-4450-8b98-0809b16290a8 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:30:46.459054 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 2cf59c2c-723c-4450-8b98-0809b16290a8 not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:30:46.467554 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-44cd32a5-bf0c-4840-9608-5c50289a01f7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:46.473171 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-44cd32a5-bf0c-4840-9608-5c50289a01f7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:46.476720 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-bb5ed9e6-f456-4872-83ec-710f791ea7ce admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:30:46.516569 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-44cd32a5-bf0c-4840-9608-5c50289a01f7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Pushing event updated for resources: {'SecurityGroup': ['ID=3eab1df2-4d7d-45a9-bcb2-293bd898fcc1,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:30:46.519741 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-44cd32a5-bf0c-4840-9608-5c50289a01f7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:30:46.520973 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-44cd32a5-bf0c-4840-9608-5c50289a01f7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:46.522166 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-44cd32a5-bf0c-4840-9608-5c50289a01f7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1778 time: 0.1955972 May 20 04:30:46.525154 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 42280) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:30:46.530196 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-04b97cd1-66c1-4d07-9668-142e3eef37ef tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Request body: {'security_group_rule': {'security_group_id': '3eab1df2-4d7d-45a9-bcb2-293bd898fcc1', 'protocol': 'tcp', 'ethertype': 'IPv4', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:30:46.536156 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-04b97cd1-66c1-4d07-9668-142e3eef37ef tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:30:46.539962 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-04b97cd1-66c1-4d07-9668-142e3eef37ef tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Usage tracker for resource:security_group_rule and tenant:198bc8ab083f4a729f32b8ef27a6ee77 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:30:46.542883 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-04b97cd1-66c1-4d07-9668-142e3eef37ef tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:30:46.549098 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-04b97cd1-66c1-4d07-9668-142e3eef37ef tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:30:46.565902 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-04b97cd1-66c1-4d07-9668-142e3eef37ef tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:46.631094 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-04b97cd1-66c1-4d07-9668-142e3eef37ef tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:46.636614 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-04b97cd1-66c1-4d07-9668-142e3eef37ef tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:46.638316 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:30:46.659245 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-04b97cd1-66c1-4d07-9668-142e3eef37ef tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=1ed7beec-bf61-4038-8229-9c5fadf0cf5b,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:30:46.661553 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-04b97cd1-66c1-4d07-9668-142e3eef37ef tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:30:46.669298 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-04b97cd1-66c1-4d07-9668-142e3eef37ef tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:46.670630 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-04b97cd1-66c1-4d07-9668-142e3eef37ef tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.1445522 May 20 04:30:46.673339 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 42286) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:30:46.678438 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-687d3152-f10d-4de0-9236-ed2053d7d48b tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Request body: {'security_group_rule': {'security_group_id': '3eab1df2-4d7d-45a9-bcb2-293bd898fcc1', 'protocol': 'icmp', 'ethertype': 'IPv4', 'direction': 'ingress'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:30:46.684518 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-687d3152-f10d-4de0-9236-ed2053d7d48b tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:30:46.688517 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-687d3152-f10d-4de0-9236-ed2053d7d48b tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Usage tracker for resource:security_group_rule and tenant:198bc8ab083f4a729f32b8ef27a6ee77 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:30:46.691537 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-687d3152-f10d-4de0-9236-ed2053d7d48b tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:30:46.697543 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-687d3152-f10d-4de0-9236-ed2053d7d48b tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:30:46.716637 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-687d3152-f10d-4de0-9236-ed2053d7d48b tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group_rule, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:46.779249 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-687d3152-f10d-4de0-9236-ed2053d7d48b tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:46.785449 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-687d3152-f10d-4de0-9236-ed2053d7d48b tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7896516'] for security_group_rule, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:46.787631 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:30:46.811363 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-687d3152-f10d-4de0-9236-ed2053d7d48b tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=92c019ce-1ce4-4426-97be-fe915d73b792,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:30:46.815691 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-687d3152-f10d-4de0-9236-ed2053d7d48b tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:30:46.823639 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-687d3152-f10d-4de0-9236-ed2053d7d48b tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group_rule, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:46.825473 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-687d3152-f10d-4de0-9236-ed2053d7d48b tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.1510382 May 20 04:30:46.829945 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 42290) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:30:46.834994 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Request body: {'floatingip': {'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:30:46.841275 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:30:46.845866 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Usage tracker for resource:floatingip and tenant:198bc8ab083f4a729f32b8ef27a6ee77 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:30:46.848862 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:30:46.854882 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:30:46.867129 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for floatingip, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:47.010106 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:47.151015 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_common [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Allocated IP 172.27.0.61 (712ce8c4-8feb-4719-bf13-e06992125cd9/e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5/ba923399-cb7e-437c-887a-3e86889e7fb9) {{(pid=91643) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:30:47.296377 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for port, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:47.331440 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:47.337420 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-81ac2e41-a193-4259-99da-80efe9fc4e50 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:30:47.404572 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 is already hosted by enough agents. {{(pid=91643) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:30:47.407330 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Resources bound (agent type, host, resource id): {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:30:47.409872 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Received port after_create {{(pid=91643) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:30:47.446771 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Pushing event updated for resources: {'Port': ['ID=ba923399-cb7e-437c-887a-3e86889e7fb9,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:30:47.450225 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.113s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:30:47.509037 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for floatingip, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:47.581268 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:47.851113 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:47.889812 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for port, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:47.896738 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:47.901523 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-b5e899a5-d8c2-4a1b-bd29-078c0c29cae0 tempest-ServerRescueNegativeTestJSON-1629517277 tempest-ServerRescueNegativeTestJSON-1629517277-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:30:48.004785 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Pushing event updated for resources: {'Port': ['ID=ba923399-cb7e-437c-887a-3e86889e7fb9,revision_number=2']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:30:48.008707 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.107s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:30:48.015463 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-8003251'] for floatingip, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:48.039589 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:48.040793 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 692 time: 1.2098825 May 20 04:30:48.231063 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-d057c678-5005-461d-be8c-e7ea10a67ee2 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Synchronizing usage tracker for tenant:198bc8ab083f4a729f32b8ef27a6ee77 on resource:security_group {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:30:48.246895 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-d057c678-5005-461d-be8c-e7ea10a67ee2 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Unset dirty status for tenant:198bc8ab083f4a729f32b8ef27a6ee77 on resource:security_group {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:30:48.248460 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-d057c678-5005-461d-be8c-e7ea10a67ee2 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?name=tempest-securitygroup--439509829&fields=id&tenant_id=198bc8ab083f4a729f32b8ef27a6ee77 HTTP/1.1" status: 200 len: 267 time: 0.0576179 May 20 04:30:48.401606 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-9b7f8aac-31c6-4f4c-b779-388176a22870 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:48.403095 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-9b7f8aac-31c6-4f4c-b779-388176a22870 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/networks?id=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d HTTP/1.1" status: 200 len: 809 time: 0.1479509 May 20 04:30:48.423838 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-b5245c84-2685-4752-b7ac-95c27d436ffa tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/quotas/198bc8ab083f4a729f32b8ef27a6ee77 HTTP/1.1" status: 200 len: 375 time: 0.0148046 May 20 04:30:48.516626 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-ef61acc1-85e8-4c0d-9996-653e3f4d1d7e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Synchronizing usage tracker for tenant:198bc8ab083f4a729f32b8ef27a6ee77 on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:30:48.533633 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-ef61acc1-85e8-4c0d-9996-653e3f4d1d7e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Unset dirty status for tenant:198bc8ab083f4a729f32b8ef27a6ee77 on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:30:48.534866 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-ef61acc1-85e8-4c0d-9996-653e3f4d1d7e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=198bc8ab083f4a729f32b8ef27a6ee77&fields=id HTTP/1.1" status: 200 len: 348 time: 0.1059306 May 20 04:30:48.678390 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-45c7872e-e47d-418d-9bb7-004cb61ab5df service neutron] 10.222.0.39 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1312737 May 20 04:30:48.814079 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-854a5b12-cfe6-4fd7-b0a3-a10bc73b9a2f service neutron] 10.222.0.39 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1276555 May 20 04:30:48.999882 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f088f2a2-7381-4ed1-9766-5c4609448af7 req-c3d887f2-a545-40ef-97c1-b0d081e4e93c tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 210 time: 0.0459571 May 20 04:30:50.137810 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-7dd8ca47-10f1-46fd-88ff-1ba3fa0e1832 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:50.139965 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-7dd8ca47-10f1-46fd-88ff-1ba3fa0e1832 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.74 "GET /v2.0/networks?id=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d HTTP/1.1" status: 200 len: 809 time: 0.1615667 May 20 04:30:50.173192 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f8702e46-8517-4617-b0c6-49c661066507 req-d2d1ec1f-7d5d-4c79-af2f-9c9dbca48124 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 210 time: 0.0532317 May 20 04:30:50.208102 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-85e4bfbd-5647-4edd-8824-b446200e8190 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.74 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=198bc8ab083f4a729f32b8ef27a6ee77 HTTP/1.1" status: 200 len: 379 time: 0.0585601 May 20 04:30:50.217766 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-c6ffb807-1557-4c96-a83a-3e0bac6e5c22 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Request body: {'port': {'device_id': 'cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'admin_state_up': True, 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'security_groups': ['3eab1df2-4d7d-45a9-bcb2-293bd898fcc1']}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:30:50.365390 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-c6ffb807-1557-4c96-a83a-3e0bac6e5c22 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:30:50.376277 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-c6ffb807-1557-4c96-a83a-3e0bac6e5c22 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:30:50.388374 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-c6ffb807-1557-4c96-a83a-3e0bac6e5c22 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Made reservation on behalf of 198bc8ab083f4a729f32b8ef27a6ee77 for: {'port': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:30:50.391869 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-c6ffb807-1557-4c96-a83a-3e0bac6e5c22 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:50.567899 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-c6ffb807-1557-4c96-a83a-3e0bac6e5c22 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Allocated IP 172.20.0.13 (49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d/4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3/c33841f1-fd78-4db6-9fee-4a5597eac2a5) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:30:50.757177 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-c6ffb807-1557-4c96-a83a-3e0bac6e5c22 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:50.772234 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-c6ffb807-1557-4c96-a83a-3e0bac6e5c22 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Transition to ACTIVE for port object c33841f1-fd78-4db6-9fee-4a5597eac2a5 will not be triggered until provisioned by entity DHCP. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:30:50.796957 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-c6ffb807-1557-4c96-a83a-3e0bac6e5c22 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:50.802656 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:30:50.870086 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-c6ffb807-1557-4c96-a83a-3e0bac6e5c22 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Network 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:30:50.873129 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-c6ffb807-1557-4c96-a83a-3e0bac6e5c22 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:30:50.876921 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-c6ffb807-1557-4c96-a83a-3e0bac6e5c22 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:30:50.906823 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c6ffb807-1557-4c96-a83a-3e0bac6e5c22 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Pushing event updated for resources: {'Port': ['ID=c33841f1-fd78-4db6-9fee-4a5597eac2a5,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:30:50.913492 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-c6ffb807-1557-4c96-a83a-3e0bac6e5c22 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.111s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:30:50.943131 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-c6ffb807-1557-4c96-a83a-3e0bac6e5c22 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:50.943758 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-c6ffb807-1557-4c96-a83a-3e0bac6e5c22 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:50.970959 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-c6ffb807-1557-4c96-a83a-3e0bac6e5c22 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 0.7571566 May 20 04:30:51.013209 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Request body: {'port': {'device_id': 'cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:30:51.103296 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-346e0333-8aec-49f8-9d32-acd4dd6ea62b None None] Provisioning for port c33841f1-fd78-4db6-9fee-4a5597eac2a5 completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:30:51.109076 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-346e0333-8aec-49f8-9d32-acd4dd6ea62b None None] Provisioning complete for port c33841f1-fd78-4db6-9fee-4a5597eac2a5 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:30:51.109557 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-346e0333-8aec-49f8-9d32-acd4dd6ea62b None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:51.133435 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:51.163393 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-346e0333-8aec-49f8-9d32-acd4dd6ea62b None None] Port c33841f1-fd78-4db6-9fee-4a5597eac2a5 cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:30:51.358122 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a41b70cb-6706-4ae1-88a6-3d82934f754d req-3042e5c7-9d90-47ec-9998-e6d0afd2b029 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:51.359634 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a41b70cb-6706-4ae1-88a6-3d82934f754d req-3042e5c7-9d90-47ec-9998-e6d0afd2b029 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 932 time: 0.0787597 May 20 04:30:51.406731 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a41b70cb-6706-4ae1-88a6-3d82934f754d req-38e45faf-4ba4-446f-826b-964629c21072 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=3eab1df2-4d7d-45a9-bcb2-293bd898fcc1&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0412784 May 20 04:30:51.475436 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id c33841f1-fd78-4db6-9fee-4a5597eac2a5 {{(pid=91644) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:30:51.482676 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:51.496844 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Transition to ACTIVE for port object c33841f1-fd78-4db6-9fee-4a5597eac2a5 will not be triggered until provisioned by entity L2. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:30:51.504431 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:51.511887 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-afb88c88-ff55-4c28-bd3d-5de837578778 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:30:51.675027 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Pushing event updated for resources: {'Port': ['ID=c33841f1-fd78-4db6-9fee-4a5597eac2a5,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:30:51.678087 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.166s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:30:51.847306 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Attempting to bind port c33841f1-fd78-4db6-9fee-4a5597eac2a5 on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:30:51.847560 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Attempting to bind port c33841f1-fd78-4db6-9fee-4a5597eac2a5 by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': 'b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 808, 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d'}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:30:51.847880 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Attempting to bind port c33841f1-fd78-4db6-9fee-4a5597eac2a5 on network 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:30:51.852770 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Port c33841f1-fd78-4db6-9fee-4a5597eac2a5 on network 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:30:51.853082 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Attempting to bind port c33841f1-fd78-4db6-9fee-4a5597eac2a5 on network 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:30:51.858210 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 30, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:30:51.858537 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Checking segment: {'id': 'b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 808, 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:30:51.858909 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Bound using segment: {'id': 'b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 808, 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d'} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:30:51.859355 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Bound port: c33841f1-fd78-4db6-9fee-4a5597eac2a5, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 808, 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d'}}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:30:51.859690 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:51.946986 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] For port c33841f1-fd78-4db6-9fee-4a5597eac2a5, host hv-cinder-78240, cleared binding levels {{(pid=91644) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:30:51.950271 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] For port c33841f1-fd78-4db6-9fee-4a5597eac2a5, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=c33841f1-fd78-4db6-9fee-4a5597eac2a5,segment=NetworkSegment(b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7),segment_id=b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7)] {{(pid=91644) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:30:51.954465 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:51.959087 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-9a836955-5f3b-4187-af10-60d0e6631e69 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:30:52.015142 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:52.015847 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:52.017312 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] 10.222.0.74 "PUT /v2.0/ports/c33841f1-fd78-4db6-9fee-4a5597eac2a5 HTTP/1.1" status: 200 len: 1094 time: 1.0087192 May 20 04:30:52.017810 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4', 'tag': 'c33841f1-fd78-4db6-9fee-4a5597eac2a5'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:30:52.048640 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Pushing event updated for resources: {'Port': ['ID=c33841f1-fd78-4db6-9fee-4a5597eac2a5,revision_number=3']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:30:52.052322 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.093s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:30:52.100492 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4', 'tag': 'c33841f1-fd78-4db6-9fee-4a5597eac2a5', 'status': 'completed', 'code': 200} May 20 04:30:52.141616 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-a813f6b9-a254-422d-a9ad-6125d239803d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:52.143100 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-a813f6b9-a254-422d-a9ad-6125d239803d service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=198bc8ab083f4a729f32b8ef27a6ee77&device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 1097 time: 0.0775681 May 20 04:30:52.221880 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-37540c20-f66e-4af7-8080-a4461b389eed service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.13&port_id=c33841f1-fd78-4db6-9fee-4a5597eac2a5 HTTP/1.1" status: 200 len: 217 time: 0.0655627 May 20 04:30:52.286169 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-637d0bc1-73c8-45ac-a654-40bfea23d9d4 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:52.287492 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-637d0bc1-73c8-45ac-a654-40bfea23d9d4 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3 HTTP/1.1" status: 200 len: 848 time: 0.0579619 May 20 04:30:52.393047 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-51d33aa1-1518-46db-94c9-48a40cdfce90 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:52.394546 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-51d33aa1-1518-46db-94c9-48a40cdfce90 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0780804 May 20 04:30:52.526437 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-11218477-35b0-4710-9ecd-be0e2792e068 service neutron] 10.222.0.74 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1244538 May 20 04:30:52.597339 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ec9b70a1-d8be-44c7-8e57-9e61380c9d7a req-454ebf2d-a9f9-4e1d-be20-0c74ea4db2a2 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:52.598780 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ec9b70a1-d8be-44c7-8e57-9e61380c9d7a req-454ebf2d-a9f9-4e1d-be20-0c74ea4db2a2 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 944 time: 0.0833671 May 20 04:30:52.645218 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ec9b70a1-d8be-44c7-8e57-9e61380c9d7a req-a32221e4-d5b0-4db4-a60c-21a97aab40b2 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=3eab1df2-4d7d-45a9-bcb2-293bd898fcc1&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0404654 May 20 04:30:52.661534 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-28663fd2-a814-4ca7-ad72-441af95238f2 req-f25fb0bc-7095-472b-b47f-f3d8260edc7d service neutron] 10.222.0.74 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1266668 May 20 04:30:52.785047 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5589f085-2362-479b-ae36-5a688ab3e717 req-ffd6ecaa-4a05-4aaf-83c8-de1a861cf386 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:52.786452 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-5589f085-2362-479b-ae36-5a688ab3e717 req-ffd6ecaa-4a05-4aaf-83c8-de1a861cf386 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=198bc8ab083f4a729f32b8ef27a6ee77&device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 1097 time: 0.0743120 May 20 04:30:52.922034 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5589f085-2362-479b-ae36-5a688ab3e717 req-b62e0d34-a237-4551-82a1-5fbac49b2066 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:52.923470 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-5589f085-2362-479b-ae36-5a688ab3e717 req-b62e0d34-a237-4551-82a1-5fbac49b2066 service neutron] 10.222.0.74 "GET /v2.0/networks?id=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d HTTP/1.1" status: 200 len: 910 time: 0.1271403 May 20 04:30:52.993718 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-5589f085-2362-479b-ae36-5a688ab3e717 req-aafa10aa-58e1-43af-8322-8647f431ca67 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.13&port_id=c33841f1-fd78-4db6-9fee-4a5597eac2a5 HTTP/1.1" status: 200 len: 217 time: 0.0626762 May 20 04:30:53.061446 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5589f085-2362-479b-ae36-5a688ab3e717 req-9a4a66b0-f786-4f79-9640-39bef44f2b18 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:53.062876 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-5589f085-2362-479b-ae36-5a688ab3e717 req-9a4a66b0-f786-4f79-9640-39bef44f2b18 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3 HTTP/1.1" status: 200 len: 848 time: 0.0607269 May 20 04:30:53.153375 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5589f085-2362-479b-ae36-5a688ab3e717 req-ab26d082-8951-42ca-9e1e-8a402f4456a9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:53.154829 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-5589f085-2362-479b-ae36-5a688ab3e717 req-ab26d082-8951-42ca-9e1e-8a402f4456a9 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0843716 May 20 04:30:53.292720 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-5589f085-2362-479b-ae36-5a688ab3e717 req-c09a17cd-5d97-417f-8bf5-bc04f1f7da07 service neutron] 10.222.0.74 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1224797 May 20 04:30:53.433043 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-5589f085-2362-479b-ae36-5a688ab3e717 req-7d0bba6b-4e95-46e7-8137-94a8272fad9c service neutron] 10.222.0.74 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1320472 May 20 04:30:53.855017 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-58ae2fc8-bb7c-47ff-abe6-4a7d7a5922d0 req-3a407232-68d8-46e3-91d2-eb67d5efc9c7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:53.856537 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-58ae2fc8-bb7c-47ff-abe6-4a7d7a5922d0 req-3a407232-68d8-46e3-91d2-eb67d5efc9c7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 944 time: 0.0911176 May 20 04:30:53.901858 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-58ae2fc8-bb7c-47ff-abe6-4a7d7a5922d0 req-aac89809-0d1a-4d4c-99c1-6d12d0d8108c tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=3eab1df2-4d7d-45a9-bcb2-293bd898fcc1&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0394483 May 20 04:30:55.103068 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-145b1cd7-fa10-445b-9dfd-39f8909fe28d req-a385ace6-c0c3-43a5-aaad-d23258d502ef tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:55.104517 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-145b1cd7-fa10-445b-9dfd-39f8909fe28d req-a385ace6-c0c3-43a5-aaad-d23258d502ef tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 944 time: 0.0800858 May 20 04:30:55.150211 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-145b1cd7-fa10-445b-9dfd-39f8909fe28d req-49062044-78d2-4156-a9f9-69829099feec tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=3eab1df2-4d7d-45a9-bcb2-293bd898fcc1&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0397470 May 20 04:30:56.338148 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d3a7488-5606-4454-ac5e-3361c77a7063 req-0b5f590f-afad-4a45-bedd-71a69baed6a9 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:56.339611 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1d3a7488-5606-4454-ac5e-3361c77a7063 req-0b5f590f-afad-4a45-bedd-71a69baed6a9 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 944 time: 0.0851007 May 20 04:30:56.386262 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1d3a7488-5606-4454-ac5e-3361c77a7063 req-cd6e2be3-3a15-45fe-b3a6-17191d3279fd tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=3eab1df2-4d7d-45a9-bcb2-293bd898fcc1&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0381393 May 20 04:30:56.438199 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device c33841f1-fd78-4db6-9fee-4a5597eac2a5 details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:30:56.579843 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port c33841f1-fd78-4db6-9fee-4a5597eac2a5, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=c33841f1-fd78-4db6-9fee-4a5597eac2a5,segment=NetworkSegment(b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7),segment_id=b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:30:56.594315 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': 'c33841f1-fd78-4db6-9fee-4a5597eac2a5', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'port_id': 'c33841f1-fd78-4db6-9fee-4a5597eac2a5', 'mac_address': 'fa:16:3e:e7:cd:da', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 808, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'ip_address': '172.20.0.13'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:30:56.650118 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:56.667751 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id c33841f1-fd78-4db6-9fee-4a5597eac2a5 {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:30:56.684105 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port c33841f1-fd78-4db6-9fee-4a5597eac2a5, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=c33841f1-fd78-4db6-9fee-4a5597eac2a5,segment=NetworkSegment(b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7),segment_id=b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:30:56.690021 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:56.692656 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:30:56.760279 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=c33841f1-fd78-4db6-9fee-4a5597eac2a5,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:30:56.767571 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:56.767958 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:56.768327 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:56.768668 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:56.768994 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:56.769298 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:56.769634 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:56.769948 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:56.770262 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:56.770572 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:56.770882 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:56.771175 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:30:56.775018 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.082s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:30:57.082850 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['c33841f1-fd78-4db6-9fee-4a5597eac2a5'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:30:57.603491 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-11f8236d-69d4-406f-ad6e-2c43f68e4fbb req-c116673a-88db-4773-a395-029afc51defa tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:57.604936 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-11f8236d-69d4-406f-ad6e-2c43f68e4fbb req-c116673a-88db-4773-a395-029afc51defa tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 945 time: 0.0954347 May 20 04:30:57.661045 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-11f8236d-69d4-406f-ad6e-2c43f68e4fbb req-647e1c4f-76c3-4e38-88c0-b1f1e800908d tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=3eab1df2-4d7d-45a9-bcb2-293bd898fcc1&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0503201 May 20 04:30:58.452389 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device c33841f1-fd78-4db6-9fee-4a5597eac2a5 up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:30:58.563912 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port c33841f1-fd78-4db6-9fee-4a5597eac2a5 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:30:58.568559 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port c33841f1-fd78-4db6-9fee-4a5597eac2a5 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:30:58.568932 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:58.692922 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:58.721334 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'c33841f1-fd78-4db6-9fee-4a5597eac2a5'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:30:58.731897 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port c33841f1-fd78-4db6-9fee-4a5597eac2a5, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=c33841f1-fd78-4db6-9fee-4a5597eac2a5,segment=NetworkSegment(b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7),segment_id=b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:30:58.737182 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:30:58.739131 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:30:58.770164 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'c33841f1-fd78-4db6-9fee-4a5597eac2a5', 'code': 200} May 20 04:30:58.807158 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=c33841f1-fd78-4db6-9fee-4a5597eac2a5,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:30:58.810086 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:30:58.872694 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c515a847-5b29-44b2-83d4-752310144ff2 req-1ef1eaea-4e42-4f03-b183-bb3347b10561 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:58.874138 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c515a847-5b29-44b2-83d4-752310144ff2 req-1ef1eaea-4e42-4f03-b183-bb3347b10561 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 946 time: 0.0898516 May 20 04:30:58.921339 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c515a847-5b29-44b2-83d4-752310144ff2 req-fab09d4d-8be4-4c0b-926c-7b15088a8fb4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=3eab1df2-4d7d-45a9-bcb2-293bd898fcc1&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0412836 May 20 04:30:59.027969 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f260e035-2a5a-41f3-9a1e-2115ac0f5851 req-480b849b-66e1-4586-acb8-42555064ef38 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:30:59.029467 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f260e035-2a5a-41f3-9a1e-2115ac0f5851 req-480b849b-66e1-4586-acb8-42555064ef38 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 1099 time: 0.0704887 May 20 04:30:59.080419 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-f260e035-2a5a-41f3-9a1e-2115ac0f5851 req-a5ae4a47-acb2-4340-9517-35bdceebca75 service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=3eab1df2-4d7d-45a9-bcb2-293bd898fcc1&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0356064 May 20 04:31:00.128153 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9e80e1e0-f192-49b9-8b89-315f29218867 req-1e638dc6-879a-4ae0-ae09-d9222c706795 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:00.129599 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9e80e1e0-f192-49b9-8b89-315f29218867 req-1e638dc6-879a-4ae0-ae09-d9222c706795 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 946 time: 0.0948403 May 20 04:31:00.175535 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9e80e1e0-f192-49b9-8b89-315f29218867 req-c38d3177-887c-4cbe-ad99-cea1007a9247 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=3eab1df2-4d7d-45a9-bcb2-293bd898fcc1&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0399368 May 20 04:31:01.374836 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-394e95bf-005d-4d9a-a292-73d2ce37188c req-a0405272-90a1-426b-b2f0-f5c5b851dc8e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:01.376194 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-394e95bf-005d-4d9a-a292-73d2ce37188c req-a0405272-90a1-426b-b2f0-f5c5b851dc8e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 946 time: 0.0881367 May 20 04:31:01.422918 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-394e95bf-005d-4d9a-a292-73d2ce37188c req-2548742e-d5b5-4cfa-b13d-453681171e90 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=3eab1df2-4d7d-45a9-bcb2-293bd898fcc1&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0409207 May 20 04:31:01.682947 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7f608360-8856-4e23-a6d5-1badae2f79a1 req-1f17d55f-bd97-439d-aae5-d2e8acc954e6 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:01.684343 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7f608360-8856-4e23-a6d5-1badae2f79a1 req-1f17d55f-bd97-439d-aae5-d2e8acc954e6 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 946 time: 0.1607709 May 20 04:31:01.697563 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 42378) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:31:01.705475 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-7b3b4f9c-f11d-4599-869c-bcab42a697aa tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Request body: {'floatingip': {'port_id': 'c33841f1-fd78-4db6-9fee-4a5597eac2a5'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:31:01.783135 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7b3b4f9c-f11d-4599-869c-bcab42a697aa tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for floatingip, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:02.404972 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7b3b4f9c-f11d-4599-869c-bcab42a697aa tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for floatingip, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:02.412128 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7b3b4f9c-f11d-4599-869c-bcab42a697aa tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-8003251'] for floatingip, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:02.581172 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.l3_db [None req-7b3b4f9c-f11d-4599-869c-bcab42a697aa tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Floating IP e600ccfa-c7c6-497d-b86a-0d2286ccdd73 associated. External IP: 172.27.0.61, port: c33841f1-fd78-4db6-9fee-4a5597eac2a5. May 20 04:31:03.048046 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-7b3b4f9c-f11d-4599-869c-bcab42a697aa tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Router 10aeccef-e762-4811-a411-a1c9817db442 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91643) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:31:03.228180 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-7b3b4f9c-f11d-4599-869c-bcab42a697aa tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:31:03.233387 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7b3b4f9c-f11d-4599-869c-bcab42a697aa tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:03.238531 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0cb37507-b0fc-471d-9a70-2955c624f358 None None] Sync routers for ids ['10aeccef-e762-4811-a411-a1c9817db442'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:31:03.248778 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0cb37507-b0fc-471d-9a70-2955c624f358 None None] Routers {'10aeccef-e762-4811-a411-a1c9817db442'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:31:03.249415 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-0cb37507-b0fc-471d-9a70-2955c624f358 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 9, "extensions": [], "floating_ips": 5, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 9, "log_agent_heartbeats": false, "routers": 9},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:30:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['10aeccef-e762-4811-a411-a1c9817db442'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:03.289943 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-7b3b4f9c-f11d-4599-869c-bcab42a697aa tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "PUT /v2.0/floatingips/e600ccfa-c7c6-497d-b86a-0d2286ccdd73 HTTP/1.1" status: 200 len: 991 time: 1.5913501 May 20 04:31:03.290579 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4', 'tag': 'c33841f1-fd78-4db6-9fee-4a5597eac2a5'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:31:03.435573 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-0cb37507-b0fc-471d-9a70-2955c624f358 None None] Set router 10aeccef-e762-4811-a411-a1c9817db442 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:31:03.478592 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4', 'tag': 'c33841f1-fd78-4db6-9fee-4a5597eac2a5', 'status': 'completed', 'code': 200} May 20 04:31:03.670457 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7dd2f617-ad04-4568-8f44-4798e6064537 req-f2d5b7cc-e3d2-45b6-a466-fafcc5e4a3b7 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:03.671881 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7dd2f617-ad04-4568-8f44-4798e6064537 req-f2d5b7cc-e3d2-45b6-a466-fafcc5e4a3b7 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=198bc8ab083f4a729f32b8ef27a6ee77&device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 1099 time: 0.1713948 May 20 04:31:03.794042 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-0cb37507-b0fc-471d-9a70-2955c624f358 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33'}], 'enable_snat': True}, 'gw_port_id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:24:36Z', 'revision_number': 4, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port': {'id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:64:1f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_gateway', 'standard_attr_id': 295, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:31Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'e600ccfa-c7c6-497d-b86a-0d2286ccdd73', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'floating_ip_address': '172.27.0.61', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '10aeccef-e762-4811-a411-a1c9817db442', 'port_id': 'c33841f1-fd78-4db6-9fee-4a5597eac2a5', 'fixed_ip_address': '172.20.0.13', 'status': 'DOWN', 'standard_attr_id': 587, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'mac_address': 'fa:16:3e:e7:cd:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:30:47Z', 'updated_at': '2021-05-20T04:31:02Z', 'revision_number': 1, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '7dc69379-dd61-4add-8e0a-4ef00a211a0a', 'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'mac_address': 'fa:16:3e:a4:49:d4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_interface', 'standard_attr_id': 307, 'fixed_ips': [{'subnet_id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:35Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 5, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'subnets': [{'id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:03.798063 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0cb37507-b0fc-471d-9a70-2955c624f358 None None] Checking router: 10aeccef-e762-4811-a411-a1c9817db442 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:31:03.804856 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7dd2f617-ad04-4568-8f44-4798e6064537 req-d4eb4c14-a380-4883-994b-13ef45dd9dd9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:03.806286 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7dd2f617-ad04-4568-8f44-4798e6064537 req-d4eb4c14-a380-4883-994b-13ef45dd9dd9 service neutron] 10.222.0.74 "GET /v2.0/networks?id=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d HTTP/1.1" status: 200 len: 910 time: 0.1275933 May 20 04:31:03.806870 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0cb37507-b0fc-471d-9a70-2955c624f358 None None] Routers {'10aeccef-e762-4811-a411-a1c9817db442'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:31:03.807515 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-0cb37507-b0fc-471d-9a70-2955c624f358 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 9, "extensions": [], "floating_ips": 5, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 9, "log_agent_heartbeats": false, "routers": 9},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:30:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['10aeccef-e762-4811-a411-a1c9817db442'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:03.914020 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7dd2f617-ad04-4568-8f44-4798e6064537 req-b2e9e739-18c8-4cff-9244-18e75b17105f service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.13&port_id=c33841f1-fd78-4db6-9fee-4a5597eac2a5 HTTP/1.1" status: 200 len: 999 time: 0.1004725 May 20 04:31:03.980612 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7dd2f617-ad04-4568-8f44-4798e6064537 req-050fcb0a-35f3-4241-b455-eb8defa91540 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:03.982092 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7dd2f617-ad04-4568-8f44-4798e6064537 req-050fcb0a-35f3-4241-b455-eb8defa91540 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3 HTTP/1.1" status: 200 len: 848 time: 0.0613275 May 20 04:31:03.991993 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-0cb37507-b0fc-471d-9a70-2955c624f358 None None] Set router 10aeccef-e762-4811-a411-a1c9817db442 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:31:04.064928 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7dd2f617-ad04-4568-8f44-4798e6064537 req-d371fcfe-e940-4c31-8604-85df9a742dcc service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:04.066419 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7dd2f617-ad04-4568-8f44-4798e6064537 req-d371fcfe-e940-4c31-8604-85df9a742dcc service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0782855 May 20 04:31:04.199899 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7dd2f617-ad04-4568-8f44-4798e6064537 req-b8b4dcc0-ce3c-4cbd-ac2d-e67eae3df3d1 service neutron] 10.222.0.74 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1263237 May 20 04:31:04.342424 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7dd2f617-ad04-4568-8f44-4798e6064537 req-d7d2b228-0ca2-4b23-92e8-6f5b58cb76e4 service neutron] 10.222.0.74 "GET /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1350987 May 20 04:31:04.355115 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-0cb37507-b0fc-471d-9a70-2955c624f358 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33'}], 'enable_snat': True}, 'gw_port_id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:24:36Z', 'revision_number': 4, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port': {'id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:64:1f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_gateway', 'standard_attr_id': 295, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:31Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'e600ccfa-c7c6-497d-b86a-0d2286ccdd73', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'floating_ip_address': '172.27.0.61', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '10aeccef-e762-4811-a411-a1c9817db442', 'port_id': 'c33841f1-fd78-4db6-9fee-4a5597eac2a5', 'fixed_ip_address': '172.20.0.13', 'status': 'DOWN', 'standard_attr_id': 587, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'mac_address': 'fa:16:3e:e7:cd:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:30:47Z', 'updated_at': '2021-05-20T04:31:02Z', 'revision_number': 1, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '7dc69379-dd61-4add-8e0a-4ef00a211a0a', 'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'mac_address': 'fa:16:3e:a4:49:d4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_interface', 'standard_attr_id': 307, 'fixed_ips': [{'subnet_id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:35Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 5, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'subnets': [{'id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:04.359877 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0cb37507-b0fc-471d-9a70-2955c624f358 None None] The sync data for ids ['10aeccef-e762-4811-a411-a1c9817db442'] in dv-cinder-782405-9-iscsi is: [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33'}], 'enable_snat': True}, 'gw_port_id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:24:36Z', 'revision_number': 4, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port': {'id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:64:1f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_gateway', 'standard_attr_id': 295, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:31Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'e600ccfa-c7c6-497d-b86a-0d2286ccdd73', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'floating_ip_address': '172.27.0.61', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '10aeccef-e762-4811-a411-a1c9817db442', 'port_id': 'c33841f1-fd78-4db6-9fee-4a5597eac2a5', 'fixed_ip_address': '172.20.0.13', 'status': 'DOWN', 'standard_attr_id': 587, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'mac_address': 'fa:16:3e:e7:cd:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:30:47Z', 'updated_at': '2021-05-20T04:31:02Z', 'revision_number': 1, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '7dc69379-dd61-4add-8e0a-4ef00a211a0a', 'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'mac_address': 'fa:16:3e:a4:49:d4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_interface', 'standard_attr_id': 307, 'fixed_ips': [{'subnet_id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:35Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 5, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'subnets': [{'id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:31:04.560372 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9dac9867-d8e5-4e31-96b1-d5e6396ea9ec None None] New status for floating IP e600ccfa-c7c6-497d-b86a-0d2286ccdd73: ACTIVE {{(pid=91646) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 20 04:31:05.622924 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:06.929557 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-844b2e95-2ce0-462f-9905-bda6dadf5b9d req-d4ccb055-e76e-45ab-ab56-57a45502ff54 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:06.930940 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-844b2e95-2ce0-462f-9905-bda6dadf5b9d req-d4ccb055-e76e-45ab-ab56-57a45502ff54 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0819533 May 20 04:31:06.980569 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-844b2e95-2ce0-462f-9905-bda6dadf5b9d req-67dcfa9b-3a85-485b-94c7-7b0133a8cbcf tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0413568 May 20 04:31:08.190882 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-27648a5c-aebf-4b67-837a-7a99c790f2ea req-15794f4d-b0ec-4903-b95e-407b9f2279dc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:08.192332 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-27648a5c-aebf-4b67-837a-7a99c790f2ea req-15794f4d-b0ec-4903-b95e-407b9f2279dc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0925541 May 20 04:31:08.236584 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-27648a5c-aebf-4b67-837a-7a99c790f2ea req-22f1941b-2822-4860-bfd1-6fac4a33c2b5 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0382609 May 20 04:31:08.713834 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9dc6e79b-dede-4578-b7df-ef18ef859945 req-3e0275b6-fe1f-4dbd-8587-4a78807d13c7 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:08.715259 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-9dc6e79b-dede-4578-b7df-ef18ef859945 req-3e0275b6-fe1f-4dbd-8587-4a78807d13c7 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=13d0e12c764f483ea457246b10c4cf7e&device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 1138 time: 0.0692008 May 20 04:31:08.755072 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4a5352a9-ff64-45ee-9ac7-0191fd23e762 req-2ad9e5f2-fc2b-4be9-9660-e0b5bfa6073d tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:08.756532 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4a5352a9-ff64-45ee-9ac7-0191fd23e762 req-2ad9e5f2-fc2b-4be9-9660-e0b5bfa6073d tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.0826473 May 20 04:31:08.804094 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4a5352a9-ff64-45ee-9ac7-0191fd23e762 req-48f2e885-69db-4b72-9fc6-a7a237c4c44d tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0415008 May 20 04:31:08.846478 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9dc6e79b-dede-4578-b7df-ef18ef859945 req-33bbd632-29ce-49e4-b498-1ed8a54c71f8 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:08.847953 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-9dc6e79b-dede-4578-b7df-ef18ef859945 req-33bbd632-29ce-49e4-b498-1ed8a54c71f8 service neutron] 10.222.0.74 "GET /v2.0/networks?id=9798d104-d4de-413f-b41d-fb9d7e8a818e HTTP/1.1" status: 200 len: 914 time: 0.1255910 May 20 04:31:08.959747 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-9dc6e79b-dede-4578-b7df-ef18ef859945 req-62c85347-b618-4a17-9bf3-0c7fb5663250 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=5f77cfe6-85e2-4a40-9a27-6ed37313ae13 HTTP/1.1" status: 200 len: 1001 time: 0.0969961 May 20 04:31:09.027747 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9dc6e79b-dede-4578-b7df-ef18ef859945 req-1b4090d6-aa94-4f78-bddc-7e3e44b646f2 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:09.029931 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-9dc6e79b-dede-4578-b7df-ef18ef859945 req-1b4090d6-aa94-4f78-bddc-7e3e44b646f2 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=c5423164-3d24-4562-871f-ab03914420dc HTTP/1.1" status: 200 len: 852 time: 0.0633256 May 20 04:31:09.095046 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e5375642-54d2-4b9f-9a00-3798941da1bb req-6c440e3f-1b81-4a9e-9726-838b67d5ca9e service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0808163 May 20 04:31:09.115444 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9dc6e79b-dede-4578-b7df-ef18ef859945 req-b4605593-8874-4ed5-8787-f693adf8dbcd service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:09.116881 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-9dc6e79b-dede-4578-b7df-ef18ef859945 req-b4605593-8874-4ed5-8787-f693adf8dbcd service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=9798d104-d4de-413f-b41d-fb9d7e8a818e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0799484 May 20 04:31:09.237497 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e5375642-54d2-4b9f-9a00-3798941da1bb req-033c0150-f4e1-455c-9b84-d1a6691dc621 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:09.238927 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e5375642-54d2-4b9f-9a00-3798941da1bb req-033c0150-f4e1-455c-9b84-d1a6691dc621 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=39febf983bf24778a300f287dbeca829&device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 1098 time: 0.0747135 May 20 04:31:09.250613 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-9dc6e79b-dede-4578-b7df-ef18ef859945 req-25bad2b1-bc74-42c1-9dcd-f9890b6a4a06 service neutron] 10.222.0.74 "GET /v2.0/networks/9798d104-d4de-413f-b41d-fb9d7e8a818e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1248503 May 20 04:31:09.385313 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-9dc6e79b-dede-4578-b7df-ef18ef859945 req-8d1c34ef-1a3f-4259-ab7f-9bbbab5d5037 service neutron] 10.222.0.74 "GET /v2.0/networks/9798d104-d4de-413f-b41d-fb9d7e8a818e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1270492 May 20 04:31:09.443734 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e5375642-54d2-4b9f-9a00-3798941da1bb req-9939906f-4c21-4fd9-8226-0dc0cd4bc9ce service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:09.445179 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e5375642-54d2-4b9f-9a00-3798941da1bb req-9939906f-4c21-4fd9-8226-0dc0cd4bc9ce service neutron] 10.222.0.74 "GET /v2.0/networks?id=e57f6e29-4637-455e-af9d-0975c45c995c HTTP/1.1" status: 200 len: 909 time: 0.1816432 May 20 04:31:09.533550 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a8f5dc96-8ed4-4a7c-9337-fef1b8c3c64e req-4d59d60f-c71f-43c7-b686-f8e368d3957d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:09.534868 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a8f5dc96-8ed4-4a7c-9337-fef1b8c3c64e req-4d59d60f-c71f-43c7-b686-f8e368d3957d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.1832552 May 20 04:31:09.578081 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e5375642-54d2-4b9f-9a00-3798941da1bb req-669fddb3-0f17-47c0-bf61-96d9ece98a79 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=b858096a-44b8-4537-a167-0e9c03347445 HTTP/1.1" status: 200 len: 1001 time: 0.1246302 May 20 04:31:09.636757 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a8f5dc96-8ed4-4a7c-9337-fef1b8c3c64e req-060a5acb-2e9c-4106-94e3-259511e180bf tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0957060 May 20 04:31:09.650287 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e5375642-54d2-4b9f-9a00-3798941da1bb req-8807b14a-8ab8-4b9f-94ad-8e678d11abc7 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:09.651856 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e5375642-54d2-4b9f-9a00-3798941da1bb req-8807b14a-8ab8-4b9f-94ad-8e678d11abc7 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=49523a89-260b-4732-abff-507d74fed4c4 HTTP/1.1" status: 200 len: 847 time: 0.0666559 May 20 04:31:09.745097 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e5375642-54d2-4b9f-9a00-3798941da1bb req-23d3533b-3cca-4acd-8b85-c71946d322b1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:09.746601 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e5375642-54d2-4b9f-9a00-3798941da1bb req-23d3533b-3cca-4acd-8b85-c71946d322b1 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=e57f6e29-4637-455e-af9d-0975c45c995c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0861356 May 20 04:31:09.886115 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e5375642-54d2-4b9f-9a00-3798941da1bb req-f0b7dad0-8381-4aec-ad67-142c5bcb1592 service neutron] 10.222.0.74 "GET /v2.0/networks/e57f6e29-4637-455e-af9d-0975c45c995c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1321988 May 20 04:31:10.071900 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e5375642-54d2-4b9f-9a00-3798941da1bb req-1055ab19-249a-4a6d-a12b-7bc2e14832d8 service neutron] 10.222.0.74 "GET /v2.0/networks/e57f6e29-4637-455e-af9d-0975c45c995c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1772354 May 20 04:31:10.087711 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bb242a11-bc40-442f-a225-25eb1bb92816 req-f3326b64-1c62-4822-ab98-df9f56b47e94 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:10.089284 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bb242a11-bc40-442f-a225-25eb1bb92816 req-f3326b64-1c62-4822-ab98-df9f56b47e94 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.1646390 May 20 04:31:10.235153 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bb242a11-bc40-442f-a225-25eb1bb92816 req-39b26ea6-07d4-47e5-878a-faf88a9f6353 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.1391928 May 20 04:31:10.866512 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-df5e2bf2-cf16-479b-9079-717bcea812f0 req-fe2979c3-c6c7-4428-9a40-0603a1917efc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:10.868002 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-df5e2bf2-cf16-479b-9079-717bcea812f0 req-fe2979c3-c6c7-4428-9a40-0603a1917efc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0934327 May 20 04:31:10.913820 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-df5e2bf2-cf16-479b-9079-717bcea812f0 req-a56aa3f4-e632-4127-9539-4a5dd6a52110 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0391860 May 20 04:31:11.449187 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a4dd9d7e-6180-44b7-922b-d72aaf9c4397 req-39d73453-4b38-496c-a52e-bdcb655110b6 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:11.450515 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a4dd9d7e-6180-44b7-922b-d72aaf9c4397 req-39d73453-4b38-496c-a52e-bdcb655110b6 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.0935867 May 20 04:31:11.496621 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a4dd9d7e-6180-44b7-922b-d72aaf9c4397 req-dc09f78f-e820-4b51-9051-b31e77730e29 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0402098 May 20 04:31:12.119126 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7bd57615-d21c-4990-8866-13a1b72b258e req-a7c1bf76-bf2c-4c53-bd01-9e958ba8ff2e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:12.120608 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7bd57615-d21c-4990-8866-13a1b72b258e req-a7c1bf76-bf2c-4c53-bd01-9e958ba8ff2e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0906768 May 20 04:31:12.169356 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7bd57615-d21c-4990-8866-13a1b72b258e req-e0598f42-c9bb-4fc6-9216-8b7c805e50d2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0428042 May 20 04:31:12.613507 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-53ec9dcb-4026-43ec-94da-8cf793f7639d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:12.694711 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-448cf45f-3374-4d86-9a81-0d9af0cb12be req-7f3206f7-3d19-45bf-aa44-572a980d46a7 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:12.696177 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-448cf45f-3374-4d86-9a81-0d9af0cb12be req-7f3206f7-3d19-45bf-aa44-572a980d46a7 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.0803123 May 20 04:31:12.746479 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-448cf45f-3374-4d86-9a81-0d9af0cb12be req-50d6f7eb-606f-4f19-a33e-ac40585d354c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0433741 May 20 04:31:13.382742 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8251e468-2872-460e-9c6c-e6d02ef12980 req-c16ed023-77c5-491b-b330-85a4ad0a0948 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:13.384096 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8251e468-2872-460e-9c6c-e6d02ef12980 req-c16ed023-77c5-491b-b330-85a4ad0a0948 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0850976 May 20 04:31:13.429188 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8251e468-2872-460e-9c6c-e6d02ef12980 req-d676fb1f-9ce8-4e1d-8269-efe5591993a7 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0393291 May 20 04:31:13.951375 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-49e7694c-b714-42a5-9cd7-7e3cc75fcb6e req-c00cdfda-ecf8-4e11-8af5-ef57df3ab14a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:13.952802 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-49e7694c-b714-42a5-9cd7-7e3cc75fcb6e req-c00cdfda-ecf8-4e11-8af5-ef57df3ab14a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.0872164 May 20 04:31:14.002967 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-49e7694c-b714-42a5-9cd7-7e3cc75fcb6e req-78f72484-532a-4b2e-972f-17a7289d3292 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0409479 May 20 04:31:14.637421 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6c75c420-8535-4786-a2f7-ecb89c272a93 req-c6bbc3ad-7f32-481d-b01e-299cce36fff9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:14.638744 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6c75c420-8535-4786-a2f7-ecb89c272a93 req-c6bbc3ad-7f32-481d-b01e-299cce36fff9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0865624 May 20 04:31:14.683997 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6c75c420-8535-4786-a2f7-ecb89c272a93 req-2a2ad264-f9c9-4bde-becf-0dc6d88d6da3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0393965 May 20 04:31:14.987589 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:15.096890 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-89e1a99b-e64a-453e-a0e0-d9058a55b130 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:15.206390 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ea7b2c6c-2b73-48b8-92ab-bbc13fd92351 req-f9c43111-e99f-4525-8159-06e5326c88cc tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:15.207915 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ea7b2c6c-2b73-48b8-92ab-bbc13fd92351 req-f9c43111-e99f-4525-8159-06e5326c88cc tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.0861800 May 20 04:31:15.259918 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ea7b2c6c-2b73-48b8-92ab-bbc13fd92351 req-28fb8974-8e11-498f-a11e-74c0c13c8b22 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0429797 May 20 04:31:15.890680 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d5945c4e-10cd-4caa-98e7-6aab2e5f9fe8 req-19fc5d63-412d-4692-a0e5-40b6f880e92e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:15.892168 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d5945c4e-10cd-4caa-98e7-6aab2e5f9fe8 req-19fc5d63-412d-4692-a0e5-40b6f880e92e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0833149 May 20 04:31:15.937217 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d5945c4e-10cd-4caa-98e7-6aab2e5f9fe8 req-454f6e9f-9334-4409-8db4-7d4f1dc85809 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0389166 May 20 04:31:16.220675 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:16.461493 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2c00fe94-02e2-41c1-9524-534cbafea919 req-58d27ad0-b184-4286-ac23-a4c41087561c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:16.462901 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2c00fe94-02e2-41c1-9524-534cbafea919 req-58d27ad0-b184-4286-ac23-a4c41087561c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.0848393 May 20 04:31:16.511469 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2c00fe94-02e2-41c1-9524-534cbafea919 req-0d737242-4e70-49a2-9072-6869d8bdd5d7 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0421667 May 20 04:31:17.153603 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f8396efc-b0fc-44f5-bc1e-7e8e2536684f req-47accf63-cdf1-4169-bcfb-b40d925cce75 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:17.155033 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f8396efc-b0fc-44f5-bc1e-7e8e2536684f req-47accf63-cdf1-4169-bcfb-b40d925cce75 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0884938 May 20 04:31:17.201944 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f8396efc-b0fc-44f5-bc1e-7e8e2536684f req-ae99fde2-6c87-4ee0-b0cb-3dd49443aed9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0410318 May 20 04:31:17.711580 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a33d3457-8969-412f-8d2f-ab61496a7702 req-74440bba-de23-4f32-b4a8-cac458199702 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:17.712989 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a33d3457-8969-412f-8d2f-ab61496a7702 req-74440bba-de23-4f32-b4a8-cac458199702 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.0821843 May 20 04:31:17.763516 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a33d3457-8969-412f-8d2f-ab61496a7702 req-eb4a8366-7660-48e6-acb8-93f9814d29e4 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0412459 May 20 04:31:17.855029 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:18.007250 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:31:18.417272 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-72d09a8c-00ca-4491-a165-9dd6183493df req-9792e211-a0f7-4cc1-812c-95c4429ac5ad tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:18.418701 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-72d09a8c-00ca-4491-a165-9dd6183493df req-9792e211-a0f7-4cc1-812c-95c4429ac5ad tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0919912 May 20 04:31:18.465564 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-72d09a8c-00ca-4491-a165-9dd6183493df req-7bab37b0-2747-433c-be72-e5e2f018cc2b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0386646 May 20 04:31:18.986685 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-73b6d20f-c52c-4d5f-bb20-80b1d1008088 req-01859eac-3a54-4c0c-961f-76ca78af2ec7 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:18.988152 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-73b6d20f-c52c-4d5f-bb20-80b1d1008088 req-01859eac-3a54-4c0c-961f-76ca78af2ec7 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.1020122 May 20 04:31:19.039431 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-73b6d20f-c52c-4d5f-bb20-80b1d1008088 req-2180e578-1eee-450c-b5de-8d55ecf03d3a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0451517 May 20 04:31:19.685564 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fa525bba-3d99-44dd-87b6-3334726f7be3 req-9b90e254-1fc5-4ed6-8f99-0d32169b4773 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:19.686988 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fa525bba-3d99-44dd-87b6-3334726f7be3 req-9b90e254-1fc5-4ed6-8f99-0d32169b4773 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0994911 May 20 04:31:19.732132 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fa525bba-3d99-44dd-87b6-3334726f7be3 req-1572f480-6a94-402b-b8aa-be6a13ceb876 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0393517 May 20 04:31:20.246166 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-218dadfb-5153-4e37-a6bf-6758c5c4345f req-79e88637-f068-476f-b386-f18aad8945dc tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:20.247554 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-218dadfb-5153-4e37-a6bf-6758c5c4345f req-79e88637-f068-476f-b386-f18aad8945dc tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.0877299 May 20 04:31:20.296687 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-218dadfb-5153-4e37-a6bf-6758c5c4345f req-a99f3f0e-37d6-4ef0-a12d-091bc6154bf8 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0433958 May 20 04:31:20.545518 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b1d353b0-0c40-42fa-a24f-0e337c3c1d7c details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:31:20.714043 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:31:20.727246 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'port_id': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c', 'mac_address': 'fa:16:3e:cf:31:63', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 847, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.9'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:31:20.783133 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:20.800043 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: ACTIVE current_port_status: BUILD port_id b1d353b0-0c40-42fa-a24f-0e337c3c1d7c {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:31:20.815451 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:31:20.820517 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:20.823582 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:20.891937 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,revision_number=6']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:20.895165 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.072s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:20.922110 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['b1d353b0-0c40-42fa-a24f-0e337c3c1d7c'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:31:20.932155 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-74f7bab6-14b7-417e-8206-f5f38454b1b9 req-0a3ff45e-1ff8-4efc-bccb-21e39d7f91c9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:20.933655 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-74f7bab6-14b7-417e-8206-f5f38454b1b9 req-0a3ff45e-1ff8-4efc-bccb-21e39d7f91c9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 944 time: 0.0797584 May 20 04:31:20.979200 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-74f7bab6-14b7-417e-8206-f5f38454b1b9 req-49a375ae-7b76-4d43-a1d6-820261d0ad44 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0396035 May 20 04:31:21.585715 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1dae7c3e-0572-41cc-91d1-afaf8fd0fe67 req-ed909a5a-f304-4880-b300-e99396216d15 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:21.587187 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1dae7c3e-0572-41cc-91d1-afaf8fd0fe67 req-ed909a5a-f304-4880-b300-e99396216d15 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.1740327 May 20 04:31:21.637615 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1dae7c3e-0572-41cc-91d1-afaf8fd0fe67 req-83cec874-ddea-45a9-8821-79be0d82b48c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0410566 May 20 04:31:22.182886 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bba4ffb5-c997-4af3-9fed-55ad5984eaad req-678e1557-ab1e-4c06-8bc1-447ad3a3fa05 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:22.184462 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bba4ffb5-c997-4af3-9fed-55ad5984eaad req-678e1557-ab1e-4c06-8bc1-447ad3a3fa05 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 944 time: 0.0923178 May 20 04:31:22.241440 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bba4ffb5-c997-4af3-9fed-55ad5984eaad req-3adb5895-a489-43b9-a8c0-2a3f2488b63c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0506864 May 20 04:31:22.560142 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b1d353b0-0c40-42fa-a24f-0e337c3c1d7c up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:31:22.627531 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:31:22.627904 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:22.744626 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:22.771996 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '74d085a4-f817-48d6-b8d8-8504cc4ca99d', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:31:22.781993 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:31:22.787965 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:22.790252 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:22.827429 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '74d085a4-f817-48d6-b8d8-8504cc4ca99d', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c', 'code': 200} May 20 04:31:22.830451 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-66cfa358-500e-43f4-8fc0-6c7a998baa1f req-331173e1-b181-4812-8859-3464fd4f9265 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:22.831940 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-66cfa358-500e-43f4-8fc0-6c7a998baa1f req-331173e1-b181-4812-8859-3464fd4f9265 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.0845416 May 20 04:31:22.859900 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,revision_number=7']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:22.862922 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.073s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:22.879602 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-66cfa358-500e-43f4-8fc0-6c7a998baa1f req-b0875560-6343-48f3-b69c-8e796ad05de7 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0416977 May 20 04:31:23.464118 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e775743d-d733-447e-bb1e-efa0e17a7518 req-3efbc1a6-b2bd-415d-b1b7-9b8ba8884ed8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:23.465511 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e775743d-d733-447e-bb1e-efa0e17a7518 req-3efbc1a6-b2bd-415d-b1b7-9b8ba8884ed8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0924807 May 20 04:31:23.512559 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e775743d-d733-447e-bb1e-efa0e17a7518 req-c0ef9c8c-a17b-4d18-b965-f7e837fab7af tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0382588 May 20 04:31:24.079004 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0f22abd-a6a2-4839-99ae-f6fefe6f7493 req-eb10f0d5-c763-459a-bb74-b824d85bcad9 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:24.080485 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b0f22abd-a6a2-4839-99ae-f6fefe6f7493 req-eb10f0d5-c763-459a-bb74-b824d85bcad9 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.0808575 May 20 04:31:24.129823 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b0f22abd-a6a2-4839-99ae-f6fefe6f7493 req-b62b8677-a4c0-4acc-89a8-a0586c51b158 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0410066 May 20 04:31:24.724770 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0d36f85f-9a1f-4f8f-9c16-57c8a74dd045 req-619fc4cb-b471-4374-b863-3cae102bb705 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:24.726212 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0d36f85f-9a1f-4f8f-9c16-57c8a74dd045 req-619fc4cb-b471-4374-b863-3cae102bb705 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0970626 May 20 04:31:24.772722 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0d36f85f-9a1f-4f8f-9c16-57c8a74dd045 req-eb1f834e-592d-4598-b6dd-2b83d3d94ec4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0395310 May 20 04:31:25.328696 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f19dcc6b-c577-4fc9-93c4-015c2250e561 req-7eae20a3-8efd-4653-b982-8c2741607be1 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:25.330131 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f19dcc6b-c577-4fc9-93c4-015c2250e561 req-7eae20a3-8efd-4653-b982-8c2741607be1 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.0864847 May 20 04:31:25.377815 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f19dcc6b-c577-4fc9-93c4-015c2250e561 req-39b3f95f-8fbc-46be-96c8-72fb002bc7d6 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0415683 May 20 04:31:25.981974 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-69294a5e-725a-4918-bb52-bdfbd1e325c0 req-fe5b2a17-af67-40a2-bffb-ed0ac52c877b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:25.983899 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-69294a5e-725a-4918-bb52-bdfbd1e325c0 req-fe5b2a17-af67-40a2-bffb-ed0ac52c877b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0970240 May 20 04:31:26.032286 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-69294a5e-725a-4918-bb52-bdfbd1e325c0 req-98588d1b-ca90-4ed7-bddc-bd6365184560 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0393891 May 20 04:31:26.590596 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b12618c9-0a49-45de-99ce-4b515a650fb1 req-40e3c2ca-9bc3-4e6d-bcc5-a88b0568b147 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:26.592442 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b12618c9-0a49-45de-99ce-4b515a650fb1 req-40e3c2ca-9bc3-4e6d-bcc5-a88b0568b147 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.0890789 May 20 04:31:26.639143 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b12618c9-0a49-45de-99ce-4b515a650fb1 req-408a3902-0732-474f-b865-32dab58bb781 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0404229 May 20 04:31:27.234925 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a69f96e0-769f-46de-b456-8455c96b239c req-b323b806-2183-47a2-9962-6824327ec10c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:27.236396 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a69f96e0-769f-46de-b456-8455c96b239c req-b323b806-2183-47a2-9962-6824327ec10c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0833817 May 20 04:31:27.287366 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a69f96e0-769f-46de-b456-8455c96b239c req-a26986fc-c564-40ef-8e98-383372d977ac tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0425375 May 20 04:31:27.839008 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8ce214e9-7806-42d0-b282-1431ee50cd83 req-dc54fa2f-b0da-4e80-8329-5f174471841d tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:27.840454 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8ce214e9-7806-42d0-b282-1431ee50cd83 req-dc54fa2f-b0da-4e80-8329-5f174471841d tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.0838244 May 20 04:31:27.887996 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8ce214e9-7806-42d0-b282-1431ee50cd83 req-808f804a-491b-4738-88f6-901f778d18cb tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0412233 May 20 04:31:28.500868 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a1eb67f3-0ad4-4226-ae6b-901aa45359d8 req-57763b88-9243-483f-86b2-54ceaac23bf4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:28.502288 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a1eb67f3-0ad4-4226-ae6b-901aa45359d8 req-57763b88-9243-483f-86b2-54ceaac23bf4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0963626 May 20 04:31:28.549256 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a1eb67f3-0ad4-4226-ae6b-901aa45359d8 req-1864313f-3cc0-4ec9-9840-030674d7323b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0382228 May 20 04:31:29.223246 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5dcbc249-240a-40ca-a922-9e4182df5ecb req-ba6eb10d-3c27-45f9-8697-3afcdfe42e82 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:29.224678 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5dcbc249-240a-40ca-a922-9e4182df5ecb req-ba6eb10d-3c27-45f9-8697-3afcdfe42e82 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.0858748 May 20 04:31:29.272234 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5dcbc249-240a-40ca-a922-9e4182df5ecb req-de2297de-3494-464e-9e31-a8b49fdb3176 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0416369 May 20 04:31:29.771456 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f68ee542-6f1f-4a46-9392-13a711d6c6a3 req-00d56600-9c5b-4066-b389-eb639f38d9c5 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:29.772924 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f68ee542-6f1f-4a46-9392-13a711d6c6a3 req-00d56600-9c5b-4066-b389-eb639f38d9c5 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.1000485 May 20 04:31:29.818379 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f68ee542-6f1f-4a46-9392-13a711d6c6a3 req-53b2c546-3e02-413a-a1e4-3ae0192d13df tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0395858 May 20 04:31:30.523044 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a8c2711a-c4fe-4944-9b31-030c582aa33d req-42b68e0d-d80f-49cc-a0ae-268384156425 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:30.524796 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a8c2711a-c4fe-4944-9b31-030c582aa33d req-42b68e0d-d80f-49cc-a0ae-268384156425 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.1028061 May 20 04:31:30.575071 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a8c2711a-c4fe-4944-9b31-030c582aa33d req-0245ffec-b017-448b-83d3-56a95ef72795 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0439687 May 20 04:31:31.018656 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bfbba1fb-b25e-47c7-bf8f-2f783abfd70a req-45ac3c2d-e8fa-49ec-b74d-b57fe88b3b5d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:31.020337 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bfbba1fb-b25e-47c7-bf8f-2f783abfd70a req-45ac3c2d-e8fa-49ec-b74d-b57fe88b3b5d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0905726 May 20 04:31:31.068864 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-bfbba1fb-b25e-47c7-bf8f-2f783abfd70a req-355ea9f8-fc43-449a-adaf-2ed7082b0d4e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0418339 May 20 04:31:31.775337 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1ef145f2-99b1-4055-b791-5349ae80d8ce req-ed5d8591-340c-4429-9c0e-542f8509573f tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:31.783129 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1ef145f2-99b1-4055-b791-5349ae80d8ce req-ed5d8591-340c-4429-9c0e-542f8509573f tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.0950372 May 20 04:31:31.830235 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1ef145f2-99b1-4055-b791-5349ae80d8ce req-12789a04-4204-406e-b87f-d0dd609e89ff tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0460658 May 20 04:31:32.279181 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f77d0ab7-faeb-43a1-928c-88426164451a req-60e4dc92-4293-4f9d-929f-c1fc943207f7 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:32.280777 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f77d0ab7-faeb-43a1-928c-88426164451a req-60e4dc92-4293-4f9d-929f-c1fc943207f7 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0812535 May 20 04:31:32.325617 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f77d0ab7-faeb-43a1-928c-88426164451a req-93c31f98-7679-4de7-81b2-e95bf50a6174 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0389698 May 20 04:31:33.104735 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-70cd04cb-6be4-4d35-af4a-bc546ecb9f73 req-15510d69-df88-40e6-bb62-a79f158cb6d7 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:33.106223 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-70cd04cb-6be4-4d35-af4a-bc546ecb9f73 req-15510d69-df88-40e6-bb62-a79f158cb6d7 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.1703444 May 20 04:31:33.155099 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-70cd04cb-6be4-4d35-af4a-bc546ecb9f73 req-8a3ab5b9-b19b-4abe-bc2d-002e7ff4466f tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0427239 May 20 04:31:33.546403 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-14abdfc8-9da6-4ba0-8e4b-0026a5935f80 req-828cb666-4d9f-4c15-997a-f03d09461181 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:33.547869 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-14abdfc8-9da6-4ba0-8e4b-0026a5935f80 req-828cb666-4d9f-4c15-997a-f03d09461181 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.1061306 May 20 04:31:33.594893 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-14abdfc8-9da6-4ba0-8e4b-0026a5935f80 req-9e518a3d-c182-4ab8-abd0-e23d682d23f0 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0403841 May 20 04:31:34.354847 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c88cc1e7-067f-4f81-b6b7-c5e3761d06e1 req-53deb6fa-65be-4f94-9e5f-db113afa06d0 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:34.356241 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c88cc1e7-067f-4f81-b6b7-c5e3761d06e1 req-53deb6fa-65be-4f94-9e5f-db113afa06d0 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.0867894 May 20 04:31:34.404345 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c88cc1e7-067f-4f81-b6b7-c5e3761d06e1 req-bca4bb2d-e109-4524-884b-6166f1aeb577 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=3a2163a0-97a4-4285-8fe2-96f2f61334ec&id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 378 time: 0.0421405 May 20 04:31:34.808494 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b74d225c-a09a-46ce-b29e-9618677d2b8f req-da01cfa3-42c6-426f-b7b1-169286d17dcf tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:34.810155 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b74d225c-a09a-46ce-b29e-9618677d2b8f req-da01cfa3-42c6-426f-b7b1-169286d17dcf tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0935078 May 20 04:31:34.822543 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 42626) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:31:34.874608 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b74d225c-a09a-46ce-b29e-9618677d2b8f req-bc0f0f84-9a18-4e83-8e8b-7ac2862db2f7 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0582001 May 20 04:31:35.367519 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-acc60b92-fcfe-4e0d-9bca-d5b0e459f7a9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Deleting port 7a1ffe9b-0e93-4df0-b2ca-608471a67ed8 {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:31:35.367962 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-acc60b92-fcfe-4e0d-9bca-d5b0e459f7a9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:35.487048 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-acc60b92-fcfe-4e0d-9bca-d5b0e459f7a9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:35.487421 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-acc60b92-fcfe-4e0d-9bca-d5b0e459f7a9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:35.614766 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-acc60b92-fcfe-4e0d-9bca-d5b0e459f7a9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Calling delete_port for 7a1ffe9b-0e93-4df0-b2ca-608471a67ed8 owned by network:floatingip {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:31:35.638894 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:35.765460 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-acc60b92-fcfe-4e0d-9bca-d5b0e459f7a9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:35.769292 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-74846be7-4875-4036-a0ad-62ddb67affbc service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:35.817066 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-acc60b92-fcfe-4e0d-9bca-d5b0e459f7a9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Pushing event deleted for resources: {'Port': ['ID=7a1ffe9b-0e93-4df0-b2ca-608471a67ed8,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:35.843624 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-acc60b92-fcfe-4e0d-9bca-d5b0e459f7a9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:35.912225 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-acc60b92-fcfe-4e0d-9bca-d5b0e459f7a9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for floatingip, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:35.912225 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.db.l3_db [None req-acc60b92-fcfe-4e0d-9bca-d5b0e459f7a9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Floating IP c8510538-1bcc-4560-92c1-c5a6e93ee028 disassociated (deleted). External IP: 172.27.0.129, port: b858096a-44b8-4537-a167-0e9c03347445. May 20 04:31:36.106449 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-db5b4ff3-1c3d-4435-92c3-e061635c1da2 req-7bcebc7a-7be9-4f84-8670-c0a61061f341 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:36.113342 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-db5b4ff3-1c3d-4435-92c3-e061635c1da2 req-7bcebc7a-7be9-4f84-8670-c0a61061f341 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.1123784 May 20 04:31:36.186370 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-db5b4ff3-1c3d-4435-92c3-e061635c1da2 req-2f6842a2-e854-4ba6-ad57-8f68d4b3a08c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0721612 May 20 04:31:36.330749 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-acc60b92-fcfe-4e0d-9bca-d5b0e459f7a9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Router 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91644) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:31:36.429480 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-f5f080a9-9d41-4a89-8605-a9acd92d724a tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-195167850&fields=id&tenant_id=0e3baa0ea24d42f3ab59e40845b350fd HTTP/1.1" status: 200 len: 267 time: 0.0562716 May 20 04:31:36.528431 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-acc60b92-fcfe-4e0d-9bca-d5b0e459f7a9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:31:36.538996 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-debe7069-dfc2-4568-8315-b99ea4355ab7 None None] Sync routers for ids ['5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:31:36.548260 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-debe7069-dfc2-4568-8315-b99ea4355ab7 None None] Routers {'5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:31:36.549242 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-debe7069-dfc2-4568-8315-b99ea4355ab7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 9, "extensions": [], "floating_ips": 5, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 9, "log_agent_heartbeats": false, "routers": 9},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:31:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:36.564349 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-acc60b92-fcfe-4e0d-9bca-d5b0e459f7a9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:36.593148 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-c82dd064-80bf-4081-8c54-3e560a3bf0ca tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:36.600040 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-c82dd064-80bf-4081-8c54-3e560a3bf0ca tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/networks?id=14c537db-01aa-4360-a348-47f94c75e03b HTTP/1.1" status: 200 len: 810 time: 0.1635225 May 20 04:31:36.616144 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-4184d483-071d-43b1-82dd-3d29694fbef1 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/quotas/0e3baa0ea24d42f3ab59e40845b350fd HTTP/1.1" status: 200 len: 375 time: 0.0151830 May 20 04:31:36.644636 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-acc60b92-fcfe-4e0d-9bca-d5b0e459f7a9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "DELETE /v2.0/floatingips/c8510538-1bcc-4560-92c1-c5a6e93ee028 HTTP/1.1" status: 204 len: 168 time: 1.8209069 May 20 04:31:36.645776 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '30d5e70e-9acd-400c-83be-01301f56a5ff', 'tag': 'b858096a-44b8-4537-a167-0e9c03347445'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:31:36.730909 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-debe7069-dfc2-4568-8315-b99ea4355ab7 None None] Set router 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:31:36.734946 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-9f7f913c-e089-47df-a7aa-a8f2e0326a38 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=0e3baa0ea24d42f3ab59e40845b350fd&fields=id HTTP/1.1" status: 200 len: 440 time: 0.1111724 May 20 04:31:36.913266 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-13df7412-c74c-426b-9c37-49d679dfdd0a service neutron] 10.222.0.39 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1770911 May 20 04:31:36.940075 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '30d5e70e-9acd-400c-83be-01301f56a5ff', 'tag': 'b858096a-44b8-4537-a167-0e9c03347445', 'status': 'completed', 'code': 200} May 20 04:31:37.100278 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-debe7069-dfc2-4568-8315-b99ea4355ab7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'name': 'tempest-TestShelveInstance-251001111-router', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100'}], 'enable_snat': True}, 'gw_port_id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:08Z', 'updated_at': '2021-05-20T04:25:15Z', 'revision_number': 4, 'project_id': '39febf983bf24778a300f287dbeca829', 'gw_port': {'id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:d3:05:8a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:10Z', 'updated_at': '2021-05-20T04:25:16Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'c7df749b-cc66-4529-a666-5c9e0a997ff8', 'name': '', 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'mac_address': 'fa:16:3e:8f:27:b0', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_interface', 'standard_attr_id': 355, 'fixed_ips': [{'subnet_id': '49523a89-260b-4732-abff-507d74fed4c4', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:14Z', 'updated_at': '2021-05-20T04:25:22Z', 'revision_number': 5, 'project_id': '39febf983bf24778a300f287dbeca829', 'subnets': [{'id': '49523a89-260b-4732-abff-507d74fed4c4', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:37.106799 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-debe7069-dfc2-4568-8315-b99ea4355ab7 None None] Checking router: 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:31:37.116541 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-debe7069-dfc2-4568-8315-b99ea4355ab7 None None] Routers {'5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:31:37.117182 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-debe7069-dfc2-4568-8315-b99ea4355ab7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 9, "extensions": [], "floating_ips": 5, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 9, "log_agent_heartbeats": false, "routers": 9},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:31:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:37.269525 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-744e599c-9e23-40de-b59c-9ed28370dfec req-9052a534-7500-4a52-a7d5-56a626c30049 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:37.278522 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-744e599c-9e23-40de-b59c-9ed28370dfec req-9052a534-7500-4a52-a7d5-56a626c30049 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.1752527 May 20 04:31:37.293209 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-ca34a25a-8238-4c6f-92c8-6a1ac46896d9 service neutron] 10.222.0.39 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3789546 May 20 04:31:37.296090 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ad7b3375-c371-4bef-8a06-a5223e7ca23d req-9a89eea3-2f9e-4a60-9ece-100aab198d93 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:37.297524 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ad7b3375-c371-4bef-8a06-a5223e7ca23d req-9a89eea3-2f9e-4a60-9ece-100aab198d93 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=39febf983bf24778a300f287dbeca829&device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 1098 time: 0.2511120 May 20 04:31:37.320074 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-debe7069-dfc2-4568-8315-b99ea4355ab7 None None] Set router 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:31:37.480647 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ad7b3375-c371-4bef-8a06-a5223e7ca23d req-ba01110f-657e-4bef-9a09-dfe3d202e82e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:37.482018 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ad7b3375-c371-4bef-8a06-a5223e7ca23d req-ba01110f-657e-4bef-9a09-dfe3d202e82e service neutron] 10.222.0.74 "GET /v2.0/networks?id=e57f6e29-4637-455e-af9d-0975c45c995c HTTP/1.1" status: 200 len: 909 time: 0.1574941 May 20 04:31:37.566892 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ad7b3375-c371-4bef-8a06-a5223e7ca23d req-55406a6d-54ae-419f-97e6-9afac7f79b77 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=b858096a-44b8-4537-a167-0e9c03347445 HTTP/1.1" status: 200 len: 217 time: 0.0754764 May 20 04:31:37.647483 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ad7b3375-c371-4bef-8a06-a5223e7ca23d req-b2dc419f-c77f-4c33-8569-8be63491f1fb service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:37.649397 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ad7b3375-c371-4bef-8a06-a5223e7ca23d req-b2dc419f-c77f-4c33-8569-8be63491f1fb service neutron] 10.222.0.74 "GET /v2.0/subnets?id=49523a89-260b-4732-abff-507d74fed4c4 HTTP/1.1" status: 200 len: 847 time: 0.0733004 May 20 04:31:37.663014 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-744e599c-9e23-40de-b59c-9ed28370dfec req-94c6e383-1574-436b-8f35-fccfa050611d tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.3819039 May 20 04:31:37.822363 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ad7b3375-c371-4bef-8a06-a5223e7ca23d req-058bcb21-83d0-4962-8542-7dde97edb923 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:37.826102 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ad7b3375-c371-4bef-8a06-a5223e7ca23d req-058bcb21-83d0-4962-8542-7dde97edb923 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=e57f6e29-4637-455e-af9d-0975c45c995c&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1630692 May 20 04:31:37.833026 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-debe7069-dfc2-4568-8315-b99ea4355ab7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'name': 'tempest-TestShelveInstance-251001111-router', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100'}], 'enable_snat': True}, 'gw_port_id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:08Z', 'updated_at': '2021-05-20T04:25:15Z', 'revision_number': 4, 'project_id': '39febf983bf24778a300f287dbeca829', 'gw_port': {'id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:d3:05:8a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:10Z', 'updated_at': '2021-05-20T04:25:16Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'c7df749b-cc66-4529-a666-5c9e0a997ff8', 'name': '', 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'mac_address': 'fa:16:3e:8f:27:b0', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_interface', 'standard_attr_id': 355, 'fixed_ips': [{'subnet_id': '49523a89-260b-4732-abff-507d74fed4c4', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:14Z', 'updated_at': '2021-05-20T04:25:22Z', 'revision_number': 5, 'project_id': '39febf983bf24778a300f287dbeca829', 'subnets': [{'id': '49523a89-260b-4732-abff-507d74fed4c4', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:37.839781 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-debe7069-dfc2-4568-8315-b99ea4355ab7 None None] The sync data for ids ['5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'] in dv-cinder-782405-9-iscsi is: [{'id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'name': 'tempest-TestShelveInstance-251001111-router', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100'}], 'enable_snat': True}, 'gw_port_id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:08Z', 'updated_at': '2021-05-20T04:25:15Z', 'revision_number': 4, 'project_id': '39febf983bf24778a300f287dbeca829', 'gw_port': {'id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:d3:05:8a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:10Z', 'updated_at': '2021-05-20T04:25:16Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'c7df749b-cc66-4529-a666-5c9e0a997ff8', 'name': '', 'network_id': 'e57f6e29-4637-455e-af9d-0975c45c995c', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'mac_address': 'fa:16:3e:8f:27:b0', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_interface', 'standard_attr_id': 355, 'fixed_ips': [{'subnet_id': '49523a89-260b-4732-abff-507d74fed4c4', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:14Z', 'updated_at': '2021-05-20T04:25:22Z', 'revision_number': 5, 'project_id': '39febf983bf24778a300f287dbeca829', 'subnets': [{'id': '49523a89-260b-4732-abff-507d74fed4c4', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:31:38.162010 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3aabb129-fe8c-4eb5-afd0-27e6301abcb2 req-8a1dc3de-6baa-48de-bf59-98c5f96e2590 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 210 time: 0.1164992 May 20 04:31:38.236868 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-19c25af1-1c91-47af-a8bf-06525ec5c98e None None] New status for floating IP c8510538-1bcc-4560-92c1-c5a6e93ee028: DOWN {{(pid=91646) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 20 04:31:38.279314 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c040fd4d-9dfc-4924-aa78-573302e29e4b req-40fd03ea-c6dd-4cb5-b5f8-91af53bc2930 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:38.280853 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c040fd4d-9dfc-4924-aa78-573302e29e4b req-40fd03ea-c6dd-4cb5-b5f8-91af53bc2930 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.4450364 May 20 04:31:38.325553 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ad7b3375-c371-4bef-8a06-a5223e7ca23d req-41d4a811-6af6-475c-bfa0-70d0945312d7 service neutron] 10.222.0.74 "GET /v2.0/networks/e57f6e29-4637-455e-af9d-0975c45c995c?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.4908786 May 20 04:31:38.381417 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c040fd4d-9dfc-4924-aa78-573302e29e4b req-860a6264-d8c4-4d61-b810-438bea09b0dc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0882142 May 20 04:31:38.573695 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ad7b3375-c371-4bef-8a06-a5223e7ca23d req-70ccca31-3712-4b2a-ba96-f38761606a19 service neutron] 10.222.0.74 "GET /v2.0/networks/e57f6e29-4637-455e-af9d-0975c45c995c?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2246068 May 20 04:31:38.938548 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b042e38c-e580-4f9e-ae4c-e36d150c3331 req-254ebcfe-a19d-42ec-b4cd-83da9ae2f5a0 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:38.940047 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b042e38c-e580-4f9e-ae4c-e36d150c3331 req-254ebcfe-a19d-42ec-b4cd-83da9ae2f5a0 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.0977752 May 20 04:31:38.988721 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b042e38c-e580-4f9e-ae4c-e36d150c3331 req-23c78ac2-8957-4500-bb13-0a2ff4e708d9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/security-groups?id=9eacb756-312a-4612-a34b-e9da9dda4f71&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0424635 May 20 04:31:39.111840 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3646335e-4cae-456c-8cd1-e989f0d8286a req-6b29e1b6-0cf2-4c5e-b6e8-d884821fe187 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:39.113322 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3646335e-4cae-456c-8cd1-e989f0d8286a req-6b29e1b6-0cf2-4c5e-b6e8-d884821fe187 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.74 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 945 time: 0.1041760 May 20 04:31:39.392459 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9d5b13f8-7fdf-48a1-ac59-a4c0c468d4f7 req-ab9af592-4472-46e1-876d-2a4e374e1fa7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 210 time: 0.0792012 May 20 04:31:39.420201 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-3646335e-4cae-456c-8cd1-e989f0d8286a req-1acb4825-0aea-4a81-a5fa-5b0280d88209 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Deleting port b858096a-44b8-4537-a167-0e9c03347445 {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:31:39.420717 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-3646335e-4cae-456c-8cd1-e989f0d8286a req-1acb4825-0aea-4a81-a5fa-5b0280d88209 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:39.581422 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-9555b673-379a-4de3-9e54-c8743b8a915b tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:39.582927 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-9555b673-379a-4de3-9e54-c8743b8a915b tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.74 "GET /v2.0/networks?id=14c537db-01aa-4360-a348-47f94c75e03b HTTP/1.1" status: 200 len: 810 time: 0.3173747 May 20 04:31:39.656532 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-3646335e-4cae-456c-8cd1-e989f0d8286a req-1acb4825-0aea-4a81-a5fa-5b0280d88209 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] For port b858096a-44b8-4537-a167-0e9c03347445, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b858096a-44b8-4537-a167-0e9c03347445,segment=NetworkSegment(c7d77442-5ac6-48cf-ab59-3f5962fc1e3e),segment_id=c7d77442-5ac6-48cf-ab59-3f5962fc1e3e)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:31:39.657323 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-3646335e-4cae-456c-8cd1-e989f0d8286a req-1acb4825-0aea-4a81-a5fa-5b0280d88209 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:39.666056 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-0fe7ec99-1dcf-4b35-afee-7ef64f0d63f5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.74 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=0e3baa0ea24d42f3ab59e40845b350fd HTTP/1.1" status: 200 len: 379 time: 0.0737450 May 20 04:31:39.684221 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-07d1522d-d8f5-4616-a208-44d870d341bc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Request body: {'port': {'device_id': '7cc291a5-7b2b-48ad-af24-f0fbbbf92685', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'admin_state_up': True, 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'security_groups': ['5a7943e2-185b-4112-9281-61ab71b1afeb']}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:31:39.849406 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-3646335e-4cae-456c-8cd1-e989f0d8286a req-1acb4825-0aea-4a81-a5fa-5b0280d88209 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Calling delete_port for b858096a-44b8-4537-a167-0e9c03347445 owned by compute:nova {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:31:39.883204 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-07d1522d-d8f5-4616-a208-44d870d341bc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:31:39.904594 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-07d1522d-d8f5-4616-a208-44d870d341bc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attempting to reserve 1 items for resource port. Total usage: 4; quota limit: 500; headroom:496 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:31:39.922154 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-07d1522d-d8f5-4616-a208-44d870d341bc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Made reservation on behalf of 0e3baa0ea24d42f3ab59e40845b350fd for: {'port': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:31:39.925854 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-07d1522d-d8f5-4616-a208-44d870d341bc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:39.928493 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-02f31553-cc73-4465-b65d-f9ca18ffb20b req-683a5bb9-7d59-4a7d-82ad-856e5252fa1d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:39.930352 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-02f31553-cc73-4465-b65d-f9ca18ffb20b req-683a5bb9-7d59-4a7d-82ad-856e5252fa1d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.2767091 May 20 04:31:40.032864 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-02f31553-cc73-4465-b65d-f9ca18ffb20b req-4c0d0453-4ad2-4265-890e-966f52aa873f tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0934684 May 20 04:31:40.085804 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-3646335e-4cae-456c-8cd1-e989f0d8286a req-1acb4825-0aea-4a81-a5fa-5b0280d88209 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:40.097679 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-15abce54-41b7-4bbf-87a2-b6e674f4f370 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:40.166769 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1acb4825-0aea-4a81-a5fa-5b0280d88209 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Pushing event deleted for resources: {'Port': ['ID=b858096a-44b8-4537-a167-0e9c03347445,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:40.177880 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-1acb4825-0aea-4a81-a5fa-5b0280d88209 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.076s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:40.196044 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-07d1522d-d8f5-4616-a208-44d870d341bc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Allocated IP 172.20.0.7 (14c537db-01aa-4360-a348-47f94c75e03b/cdea475c-2929-4ae4-9271-87cc2fb9a378/fb2044c2-038a-473c-968f-29ae50d3c1a4) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:31:40.318772 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-42bdf579-4b05-4ee4-bd4f-f30f4b54b421 req-79909c84-936e-423b-bcba-d9f8b33b6fa7 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 210 time: 0.1396291 May 20 04:31:40.320096 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [req-3646335e-4cae-456c-8cd1-e989f0d8286a req-1acb4825-0aea-4a81-a5fa-5b0280d88209 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] No DVR routers for this DVR port b858096a-44b8-4537-a167-0e9c03347445 on host hv-cinder-78240 {{(pid=91643) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:31:40.323052 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-3646335e-4cae-456c-8cd1-e989f0d8286a req-1acb4825-0aea-4a81-a5fa-5b0280d88209 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:40.332773 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '30d5e70e-9acd-400c-83be-01301f56a5ff', 'name': 'network-vif-deleted', 'tag': 'b858096a-44b8-4537-a167-0e9c03347445'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:31:40.338379 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3646335e-4cae-456c-8cd1-e989f0d8286a req-1acb4825-0aea-4a81-a5fa-5b0280d88209 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.74 "DELETE /v2.0/ports/b858096a-44b8-4537-a167-0e9c03347445 HTTP/1.1" status: 204 len: 173 time: 1.2171669 May 20 04:31:40.437333 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '30d5e70e-9acd-400c-83be-01301f56a5ff', 'name': 'network-vif-deleted', 'tag': 'b858096a-44b8-4537-a167-0e9c03347445', 'status': 'completed', 'code': 200} May 20 04:31:40.521853 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-07d1522d-d8f5-4616-a208-44d870d341bc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:40.556559 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-07d1522d-d8f5-4616-a208-44d870d341bc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Transition to ACTIVE for port object fb2044c2-038a-473c-968f-29ae50d3c1a4 will not be triggered until provisioned by entity DHCP. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:31:40.575447 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7c49207b-3675-4534-a999-9e5f83166e34 req-1ccbc54b-f94c-467f-8814-f798a71da74c tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 210 time: 0.0519633 May 20 04:31:40.611917 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-07d1522d-d8f5-4616-a208-44d870d341bc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:40.618068 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:40.695703 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-07d1522d-d8f5-4616-a208-44d870d341bc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Network 14c537db-01aa-4360-a348-47f94c75e03b is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:31:40.700768 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-07d1522d-d8f5-4616-a208-44d870d341bc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:31:40.705093 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-07d1522d-d8f5-4616-a208-44d870d341bc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:31:40.767748 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-07d1522d-d8f5-4616-a208-44d870d341bc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event updated for resources: {'Port': ['ID=fb2044c2-038a-473c-968f-29ae50d3c1a4,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:40.776406 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-07d1522d-d8f5-4616-a208-44d870d341bc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.158s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:40.800629 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-07d1522d-d8f5-4616-a208-44d870d341bc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:40.801268 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-07d1522d-d8f5-4616-a208-44d870d341bc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:40.861719 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-07d1522d-d8f5-4616-a208-44d870d341bc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.1898370 May 20 04:31:40.880831 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Request body: {'port': {'device_id': '7cc291a5-7b2b-48ad-af24-f0fbbbf92685', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:31:41.050693 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:41.110163 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-9dd79ca7-a1df-428b-bc19-de71e3800e7e None None] Provisioning for port fb2044c2-038a-473c-968f-29ae50d3c1a4 completed by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:31:41.123862 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-9dd79ca7-a1df-428b-bc19-de71e3800e7e None None] Provisioning complete for port fb2044c2-038a-473c-968f-29ae50d3c1a4 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:31:41.124566 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-9dd79ca7-a1df-428b-bc19-de71e3800e7e None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:41.238168 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-9dd79ca7-a1df-428b-bc19-de71e3800e7e None None] Port fb2044c2-038a-473c-968f-29ae50d3c1a4 cannot update to ACTIVE because it is not bound. {{(pid=91645) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 20 04:31:41.411849 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-df08cdc9-edbd-4f1e-9425-2c10ae9d0a06 req-f7f3f738-cb87-4df5-8ac5-d640c5a52a2b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:41.414041 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-df08cdc9-edbd-4f1e-9425-2c10ae9d0a06 req-f7f3f738-cb87-4df5-8ac5-d640c5a52a2b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.1705501 May 20 04:31:41.551867 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id fb2044c2-038a-473c-968f-29ae50d3c1a4 {{(pid=91644) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:31:41.560868 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:41.579661 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Transition to ACTIVE for port object fb2044c2-038a-473c-968f-29ae50d3c1a4 will not be triggered until provisioned by entity L2. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:31:41.585928 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:41.597923 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-5b3bb56b-b1df-4274-8ce5-54abfcf4e9e1 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:41.688693 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-df08cdc9-edbd-4f1e-9425-2c10ae9d0a06 req-ee2eb522-20e1-4c1c-8c8d-808daec4ae0d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.2611389 May 20 04:31:41.888697 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Pushing event updated for resources: {'Port': ['ID=fb2044c2-038a-473c-968f-29ae50d3c1a4,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:41.901020 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.303s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:41.940705 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Attempting to bind port fb2044c2-038a-473c-968f-29ae50d3c1a4 on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:31:41.941051 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Attempting to bind port fb2044c2-038a-473c-968f-29ae50d3c1a4 by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:31:41.941364 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Attempting to bind port fb2044c2-038a-473c-968f-29ae50d3c1a4 on network 14c537db-01aa-4360-a348-47f94c75e03b {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:31:41.946594 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Port fb2044c2-038a-473c-968f-29ae50d3c1a4 on network 14c537db-01aa-4360-a348-47f94c75e03b not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:31:41.946911 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Attempting to bind port fb2044c2-038a-473c-968f-29ae50d3c1a4 on network 14c537db-01aa-4360-a348-47f94c75e03b {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:31:41.952738 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 31, 36), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:31:41.953075 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Checking segment: {'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:31:41.953457 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Bound using segment: {'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:31:41.954026 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Bound port: fb2044c2-038a-473c-968f-29ae50d3c1a4, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '05406cc0-328f-4fba-b043-20b2b57b274d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 841, 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b'}}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:31:41.954373 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:41.989862 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-59f83f4a-2e0c-4040-a917-17cdb560659e req-e3c453e6-1826-4e35-8151-2059b6123645 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Synchronizing usage tracker for tenant:39febf983bf24778a300f287dbeca829 on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:31:42.017049 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-59f83f4a-2e0c-4040-a917-17cdb560659e req-e3c453e6-1826-4e35-8151-2059b6123645 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Unset dirty status for tenant:39febf983bf24778a300f287dbeca829 on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:31:42.029326 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-59f83f4a-2e0c-4040-a917-17cdb560659e req-e3c453e6-1826-4e35-8151-2059b6123645 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "GET /v2.0/ports?device_id=30d5e70e-9acd-400c-83be-01301f56a5ff HTTP/1.1" status: 200 len: 210 time: 0.3233185 May 20 04:31:42.049341 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f3856cae-609a-42c5-97bb-f9c547707b69 req-6711be55-9038-4565-8365-97466395d28e tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:42.052094 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f3856cae-609a-42c5-97bb-f9c547707b69 req-6711be55-9038-4565-8365-97466395d28e tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 943 time: 0.2194214 May 20 04:31:42.094256 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] For port fb2044c2-038a-473c-968f-29ae50d3c1a4, host hv-cinder-78240, cleared binding levels {{(pid=91644) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:31:42.097735 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] For port fb2044c2-038a-473c-968f-29ae50d3c1a4, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=fb2044c2-038a-473c-968f-29ae50d3c1a4,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91644) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:31:42.104953 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f3856cae-609a-42c5-97bb-f9c547707b69 req-38ab8f6e-b832-491b-a08a-6e48e432409b tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0459950 May 20 04:31:42.105303 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:42.115037 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:42.171858 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:42.172476 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:42.173864 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] 10.222.0.74 "PUT /v2.0/ports/fb2044c2-038a-473c-968f-29ae50d3c1a4 HTTP/1.1" status: 200 len: 1093 time: 1.3039942 May 20 04:31:42.174356 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '7cc291a5-7b2b-48ad-af24-f0fbbbf92685', 'tag': 'fb2044c2-038a-473c-968f-29ae50d3c1a4'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:31:42.295368 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '7cc291a5-7b2b-48ad-af24-f0fbbbf92685', 'tag': 'fb2044c2-038a-473c-968f-29ae50d3c1a4', 'status': 'completed', 'code': 200} May 20 04:31:42.311138 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Pushing event updated for resources: {'Port': ['ID=fb2044c2-038a-473c-968f-29ae50d3c1a4,revision_number=3']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:42.317843 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.203s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:42.359518 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-bc12e630-24f0-48f9-8c05-7079bac18ac5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:42.361051 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-bc12e630-24f0-48f9-8c05-7079bac18ac5 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=0e3baa0ea24d42f3ab59e40845b350fd&device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 1096 time: 0.1203978 May 20 04:31:42.539598 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-d1dfcef1-2624-4f85-a574-d304cf52d1c2 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=fb2044c2-038a-473c-968f-29ae50d3c1a4 HTTP/1.1" status: 200 len: 217 time: 0.1703775 May 20 04:31:42.613647 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-3903a9df-e824-472f-a7e1-f04046b95dad service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:42.615992 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-3903a9df-e824-472f-a7e1-f04046b95dad service neutron] 10.222.0.74 "GET /v2.0/subnets?id=cdea475c-2929-4ae4-9271-87cc2fb9a378 HTTP/1.1" status: 200 len: 849 time: 0.0671875 May 20 04:31:42.620074 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b858096a-44b8-4537-a167-0e9c03347445 no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:31:42.650232 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-eb6dfbac-00ba-412b-9fff-e587602c9cb8 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:42.660432 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: b858096a-44b8-4537-a167-0e9c03347445 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:31:42.661076 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b858096a-44b8-4537-a167-0e9c03347445 not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:31:42.702677 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-93436618-63f1-4a22-a339-48d52551869d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:42.704672 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-93436618-63f1-4a22-a339-48d52551869d service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=14c537db-01aa-4360-a348-47f94c75e03b&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0823565 May 20 04:31:42.854414 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-2e78e43f-cf1e-4934-a018-95d385fd0a3e service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1430449 May 20 04:31:42.886554 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8d943d20-8bbf-4a11-85ac-0489dec78784 req-ab2c80ff-5c01-4493-90cc-337f5ac028a8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:42.895578 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8d943d20-8bbf-4a11-85ac-0489dec78784 req-ab2c80ff-5c01-4493-90cc-337f5ac028a8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.0975394 May 20 04:31:42.948699 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8d943d20-8bbf-4a11-85ac-0489dec78784 req-e3f660dd-40a1-491a-8fb5-958bd16fd465 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0521770 May 20 04:31:43.017065 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-38f74ce3-9140-4e6a-9205-31db6da97385 req-277461e8-83de-4433-a789-1acdbe8124e5 service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1547821 May 20 04:31:43.159786 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d9da3471-3634-4274-85a3-8125384d94ef req-0e52144d-55d5-4c4d-b3d1-105d465fc3fd service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:43.161521 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-d9da3471-3634-4274-85a3-8125384d94ef req-0e52144d-55d5-4c4d-b3d1-105d465fc3fd service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=0e3baa0ea24d42f3ab59e40845b350fd&device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 1096 time: 0.0859630 May 20 04:31:43.175555 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 42718) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:31:43.330772 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-0dd35998-56f5-4839-bf64-22a0fa5ba10d tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:43.337252 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d9da3471-3634-4274-85a3-8125384d94ef req-70858abf-dae5-494f-a3be-c97bbdeae63b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:43.338912 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-d9da3471-3634-4274-85a3-8125384d94ef req-70858abf-dae5-494f-a3be-c97bbdeae63b service neutron] 10.222.0.74 "GET /v2.0/networks?id=14c537db-01aa-4360-a348-47f94c75e03b HTTP/1.1" status: 200 len: 911 time: 0.1710305 May 20 04:31:43.372092 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-0dd35998-56f5-4839-bf64-22a0fa5ba10d tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:43.415465 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-df8b0733-486f-4d4f-bc75-98342b87d7f3 req-1a67c44e-9a6c-4582-a39f-17de595d22ca tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:43.415737 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-d9da3471-3634-4274-85a3-8125384d94ef req-d61552c5-dec7-4f6b-8b83-c7fc2c4235e5 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=fb2044c2-038a-473c-968f-29ae50d3c1a4 HTTP/1.1" status: 200 len: 217 time: 0.0670111 May 20 04:31:43.415841 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-df8b0733-486f-4d4f-bc75-98342b87d7f3 req-1a67c44e-9a6c-4582-a39f-17de595d22ca tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 943 time: 0.1533523 May 20 04:31:43.488967 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d9da3471-3634-4274-85a3-8125384d94ef req-068b3091-d256-46b9-9dec-214664bf642a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:43.490392 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-d9da3471-3634-4274-85a3-8125384d94ef req-068b3091-d256-46b9-9dec-214664bf642a service neutron] 10.222.0.74 "GET /v2.0/subnets?id=cdea475c-2929-4ae4-9271-87cc2fb9a378 HTTP/1.1" status: 200 len: 849 time: 0.0710604 May 20 04:31:43.491795 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-df8b0733-486f-4d4f-bc75-98342b87d7f3 req-a4311059-4560-4a4c-8833-7bc44f812dad tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0655591 May 20 04:31:43.522062 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-0dd35998-56f5-4839-bf64-22a0fa5ba10d tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:43.528729 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-d7fc5fcf-d139-4e50-b5cd-edec41470990 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:43.545716 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0dd35998-56f5-4839-bf64-22a0fa5ba10d tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=9eacb756-312a-4612-a34b-e9da9dda4f71,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:43.549317 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-0dd35998-56f5-4839-bf64-22a0fa5ba10d tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.021s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:43.554825 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-0dd35998-56f5-4839-bf64-22a0fa5ba10d tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:43.558731 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-0dd35998-56f5-4839-bf64-22a0fa5ba10d tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] 10.222.0.39 "DELETE /v2.0/security-groups/9eacb756-312a-4612-a34b-e9da9dda4f71 HTTP/1.1" status: 204 len: 168 time: 0.3788755 May 20 04:31:43.589240 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d9da3471-3634-4274-85a3-8125384d94ef req-8a4f34db-f61f-4571-bc44-3cf994ea34c5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:43.593473 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-d9da3471-3634-4274-85a3-8125384d94ef req-8a4f34db-f61f-4571-bc44-3cf994ea34c5 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=14c537db-01aa-4360-a348-47f94c75e03b&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0966096 May 20 04:31:43.632362 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 42734) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:31:43.744226 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-d9da3471-3634-4274-85a3-8125384d94ef req-aa69430d-b6b9-4ec5-a880-9aa1748bbc0e service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1430049 May 20 04:31:43.881700 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-d9da3471-3634-4274-85a3-8125384d94ef req-e9e5f455-abd4-4f7d-bc9b-c6da25a55b4c service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1281691 May 20 04:31:44.247847 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ecaa087e-b221-46ec-ab8f-8393aaa30986 req-98cbdaff-5256-44f9-a1aa-51de0a14831f tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:44.249166 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ecaa087e-b221-46ec-ab8f-8393aaa30986 req-98cbdaff-5256-44f9-a1aa-51de0a14831f tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.1821733 May 20 04:31:44.311641 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ecaa087e-b221-46ec-ab8f-8393aaa30986 req-ce5eb803-4ea3-42d0-8182-d33fcfa0baf6 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0559616 May 20 04:31:44.394494 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-8003145'] for router_interface, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:44.841301 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-18c60fc3-62e7-4697-a94c-34b8e9d663ec req-998db32f-257d-4fc9-b320-75fe2637239a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:44.843244 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-18c60fc3-62e7-4697-a94c-34b8e9d663ec req-998db32f-257d-4fc9-b320-75fe2637239a service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=9fafdf3a29024ac588e5b24719735677&device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 1098 time: 0.0773115 May 20 04:31:44.867225 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6cb9054f-0515-4bcd-aa5b-adab35df0c06 req-43666f42-6088-4082-8ef3-18e875eb6e05 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:44.868676 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6cb9054f-0515-4bcd-aa5b-adab35df0c06 req-43666f42-6088-4082-8ef3-18e875eb6e05 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 943 time: 0.2394583 May 20 04:31:44.962329 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6cb9054f-0515-4bcd-aa5b-adab35df0c06 req-11d2be7f-e085-4ff5-9d4f-40d3b804ce0f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0870681 May 20 04:31:44.980388 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-18c60fc3-62e7-4697-a94c-34b8e9d663ec req-f3178cd1-8bb8-4437-908c-d49989433d08 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:44.981900 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-18c60fc3-62e7-4697-a94c-34b8e9d663ec req-f3178cd1-8bb8-4437-908c-d49989433d08 service neutron] 10.222.0.74 "GET /v2.0/networks?id=64bcea8a-6d35-406b-a823-fa1185008174 HTTP/1.1" status: 200 len: 918 time: 0.1317577 May 20 04:31:45.000853 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:45.027119 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Deleting port c7df749b-cc66-4529-a666-5c9e0a997ff8 {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:31:45.027487 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:45.098630 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-18c60fc3-62e7-4697-a94c-34b8e9d663ec req-29173d05-5611-4293-bec1-b7e626fb5b11 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c HTTP/1.1" status: 200 len: 1000 time: 0.1074588 May 20 04:31:45.102361 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-08f7aff7-c4ba-468d-9ac3-37298857a598 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:45.161560 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] For port c7df749b-cc66-4529-a666-5c9e0a997ff8, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=c7df749b-cc66-4529-a666-5c9e0a997ff8,segment=NetworkSegment(c7d77442-5ac6-48cf-ab59-3f5962fc1e3e),segment_id=c7d77442-5ac6-48cf-ab59-3f5962fc1e3e)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:31:45.161874 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:45.251870 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-18c60fc3-62e7-4697-a94c-34b8e9d663ec req-870f515e-a480-402b-84ae-fad8693ab6c5 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:45.253294 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-18c60fc3-62e7-4697-a94c-34b8e9d663ec req-870f515e-a480-402b-84ae-fad8693ab6c5 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=203a3962-2282-4f51-ac92-c9695b5c0cfe HTTP/1.1" status: 200 len: 856 time: 0.1484571 May 20 04:31:45.286704 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Calling delete_port for c7df749b-cc66-4529-a666-5c9e0a997ff8 owned by network:router_interface {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:31:45.327449 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-18c60fc3-62e7-4697-a94c-34b8e9d663ec req-625af228-fdf6-46f2-9600-51b3ef74f2e5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:45.328915 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-18c60fc3-62e7-4697-a94c-34b8e9d663ec req-625af228-fdf6-46f2-9600-51b3ef74f2e5 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=64bcea8a-6d35-406b-a823-fa1185008174&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0696657 May 20 04:31:45.459525 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-18c60fc3-62e7-4697-a94c-34b8e9d663ec req-d8d64ff7-ced0-4c66-9717-1207994d7c64 service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1235964 May 20 04:31:45.572120 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eae1cdaf-ed14-4ba5-a143-b2b618f54e33 req-016fb871-5824-4008-96e4-391f225d25d5 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:45.573970 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-eae1cdaf-ed14-4ba5-a143-b2b618f54e33 req-016fb871-5824-4008-96e4-391f225d25d5 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 945 time: 0.1384256 May 20 04:31:45.593918 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-18c60fc3-62e7-4697-a94c-34b8e9d663ec req-e0412dda-c890-4f03-a579-78d4b1b26957 service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1270585 May 20 04:31:45.687165 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-eae1cdaf-ed14-4ba5-a143-b2b618f54e33 req-5fcc5721-225d-43e4-9825-dce973ce326a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1048086 May 20 04:31:45.699590 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:45.703313 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-3959a328-08c8-4edb-b86d-9d78a2929cd2 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:45.744728 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Pushing event deleted for resources: {'Port': ['ID=c7df749b-cc66-4529-a666-5c9e0a997ff8,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:45.781874 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:45.782229 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:45.782607 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:45.782957 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:45.783333 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:45.783673 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:45.784029 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:45.784377 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:45.784712 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:45.785069 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:45.785409 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:45.785748 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:45.793415 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.090s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:46.024663 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-7924922', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-8054745'] for router_interface, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:46.215359 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:46.434199 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4b6b3072-5785-4957-ade7-c5eb67a172f3 req-9d533f2c-a272-4989-9a31-836df22fadc2 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:46.435658 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4b6b3072-5785-4957-ade7-c5eb67a172f3 req-9d533f2c-a272-4989-9a31-836df22fadc2 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 943 time: 0.3297250 May 20 04:31:46.560254 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4b6b3072-5785-4957-ade7-c5eb67a172f3 req-b9d6586d-7b33-4561-ad21-699cf4183b68 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1160367 May 20 04:31:46.582240 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Router 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91643) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:31:46.629609 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b1d353b0-0c40-42fa-a24f-0e337c3c1d7c no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:31:46.743678 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:46.761070 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:31:46.773623 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '74d085a4-f817-48d6-b8d8-8504cc4ca99d', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:31:46.782075 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e92b6bf8-d325-4c9c-9cd1-fc3fc8183bd7 None None] Sync routers for ids ['5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:31:46.782270 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-7bd8115a-d28a-4d56-970e-7f12a5b225cb admin admin] 10.222.0.39 "PUT /v2.0/routers/5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 3.1452205 May 20 04:31:46.782698 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 42758) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:31:46.792545 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:31:46.798671 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e92b6bf8-d325-4c9c-9cd1-fc3fc8183bd7 None None] Routers {'5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:31:46.799695 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-e92b6bf8-d325-4c9c-9cd1-fc3fc8183bd7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 9, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 9, "log_agent_heartbeats": false, "routers": 9},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:31:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:46.873088 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '74d085a4-f817-48d6-b8d8-8504cc4ca99d', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c', 'code': 200} May 20 04:31:46.874211 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:46.880689 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:46.890100 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a2fee417-d6bc-41bc-b85d-f05aeb01b6dc req-6125306e-989d-445a-92b5-5189a9d2e2bb tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:46.891700 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a2fee417-d6bc-41bc-b85d-f05aeb01b6dc req-6125306e-989d-445a-92b5-5189a9d2e2bb tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0854940 May 20 04:31:46.940449 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a2fee417-d6bc-41bc-b85d-f05aeb01b6dc req-3e2bf734-38b1-4c4e-a35a-03bfc4a100db tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0430346 May 20 04:31:46.984691 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Notify callbacks [] for router, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:47.033991 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,revision_number=8']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:47.038816 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.158s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:47.040571 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-e92b6bf8-d325-4c9c-9cd1-fc3fc8183bd7 None None] Set router 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:31:47.161983 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:47.234414 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Deleting port 37393b04-8a64-4737-953b-1c02d4efce9f {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:31:47.234809 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:47.331502 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-e92b6bf8-d325-4c9c-9cd1-fc3fc8183bd7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'name': 'tempest-TestShelveInstance-251001111-router', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100'}], 'enable_snat': True}, 'gw_port_id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:08Z', 'updated_at': '2021-05-20T04:31:45Z', 'revision_number': 5, 'project_id': '39febf983bf24778a300f287dbeca829', 'gw_port': {'id': '37393b04-8a64-4737-953b-1c02d4efce9f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:d3:05:8a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'device_owner': 'network:router_gateway', 'standard_attr_id': 354, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.100', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:10Z', 'updated_at': '2021-05-20T04:25:16Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:47.335432 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e92b6bf8-d325-4c9c-9cd1-fc3fc8183bd7 None None] Checking router: 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:31:47.345347 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e92b6bf8-d325-4c9c-9cd1-fc3fc8183bd7 None None] Routers {'5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:31:47.346013 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-e92b6bf8-d325-4c9c-9cd1-fc3fc8183bd7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 9, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 9, "log_agent_heartbeats": false, "routers": 9},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:31:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:47.356693 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] For port 37393b04-8a64-4737-953b-1c02d4efce9f, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=37393b04-8a64-4737-953b-1c02d4efce9f,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:31:47.357010 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:47.478875 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Calling delete_port for 37393b04-8a64-4737-953b-1c02d4efce9f owned by network:router_gateway {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:31:47.543239 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-e92b6bf8-d325-4c9c-9cd1-fc3fc8183bd7 None None] Set router 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 gateway port host: None {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:31:47.620651 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:47.626110 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-32322b86-9840-4bf9-b2bf-2072a5053003 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:47.651169 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-e92b6bf8-d325-4c9c-9cd1-fc3fc8183bd7 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'name': 'tempest-TestShelveInstance-251001111-router', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:08Z', 'updated_at': '2021-05-20T04:31:47Z', 'revision_number': 6, 'project_id': '39febf983bf24778a300f287dbeca829', 'gw_port_host': None}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:47.655490 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e92b6bf8-d325-4c9c-9cd1-fc3fc8183bd7 None None] The sync data for ids ['5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66'] in dv-cinder-782405-9-iscsi is: [{'id': '5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66', 'name': 'tempest-TestShelveInstance-251001111-router', 'tenant_id': '39febf983bf24778a300f287dbeca829', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:08Z', 'updated_at': '2021-05-20T04:31:47Z', 'revision_number': 6, 'project_id': '39febf983bf24778a300f287dbeca829', 'gw_port_host': None}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:31:47.697456 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Pushing event deleted for resources: {'Port': ['ID=37393b04-8a64-4737-953b-1c02d4efce9f,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:47.717766 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:47.718130 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:47.718487 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:47.718835 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:47.719177 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:47.719547 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:47.719889 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:47.720225 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:47.720566 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:47.720898 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:47.721230 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:47.721620 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:31:47.752083 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.126s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:47.753629 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38d2ed65-229c-4e63-9c63-f77f0eb813cf req-fb66c35e-d418-47ad-ba0c-f4e106027eef tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:47.755086 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-38d2ed65-229c-4e63-9c63-f77f0eb813cf req-fb66c35e-d418-47ad-ba0c-f4e106027eef tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 943 time: 0.0864778 May 20 04:31:47.778982 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-8053346'] for router_gateway, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:47.807806 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-38d2ed65-229c-4e63-9c63-f77f0eb813cf req-2805196f-f12f-421c-8091-29ddfca951a5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0455544 May 20 04:31:47.855100 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:48.055277 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-8071641', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-260698'] for router, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:48.059864 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:48.093391 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-8056629'] for router, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:48.095945 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Fanout notify agent at l3_agent the message router_deleted on router 5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 {{(pid=91644) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 20 04:31:48.111295 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] Notify callbacks [] for router, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:48.111295 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-ffab71ee-25a1-4d70-9539-92c961b38555 admin admin] 10.222.0.39 "DELETE /v2.0/routers/5b0cf8bc-8b71-4fdc-b84e-942ba3ca9c66 HTTP/1.1" status: 204 len: 168 time: 1.3268542 May 20 04:31:48.113634 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 42768) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:31:48.123859 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device c7df749b-cc66-4529-a666-5c9e0a997ff8 no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:31:48.170012 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: c7df749b-cc66-4529-a666-5c9e0a997ff8 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:31:48.170457 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device c7df749b-cc66-4529-a666-5c9e0a997ff8 not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:31:48.171113 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6ff3e0ba-7e3a-4005-83d2-87bc0556e976 req-a68efe7f-dc8f-481c-8a10-63e9f80bd2f0 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:48.173959 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6ff3e0ba-7e3a-4005-83d2-87bc0556e976 req-a68efe7f-dc8f-481c-8a10-63e9f80bd2f0 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.1086383 May 20 04:31:48.192968 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_v2 [None req-016da6ad-d061-4247-8c9f-02269b3e8172 admin admin] Deleting subnet 49523a89-260b-4732-abff-507d74fed4c4 {{(pid=91644) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 20 04:31:48.195905 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-016da6ad-d061-4247-8c9f-02269b3e8172 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:48.218998 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6ff3e0ba-7e3a-4005-83d2-87bc0556e976 req-4681e006-6ef4-455b-9c22-0fa91378734b tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0387816 May 20 04:31:48.245323 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-016da6ad-d061-4247-8c9f-02269b3e8172 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-853257'] for subnet, precommit_delete_associations {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:48.364046 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-016da6ad-d061-4247-8c9f-02269b3e8172 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-7786687'] for subnet, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:48.650176 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-016da6ad-d061-4247-8c9f-02269b3e8172 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-7786632', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-1988925', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-1810229'] for subnet, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:48.659989 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:48.699195 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-016da6ad-d061-4247-8c9f-02269b3e8172 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=49523a89-260b-4732-abff-507d74fed4c4,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:48.700379 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-016da6ad-d061-4247-8c9f-02269b3e8172 admin admin] Notify callbacks [] for subnet, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:48.706328 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-016da6ad-d061-4247-8c9f-02269b3e8172 admin admin] 10.222.0.39 "DELETE /v2.0/subnets/49523a89-260b-4732-abff-507d74fed4c4 HTTP/1.1" status: 204 len: 168 time: 0.5915775 May 20 04:31:48.707325 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-016da6ad-d061-4247-8c9f-02269b3e8172 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:48.708794 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 42784) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:31:48.711657 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-985855d9-437e-49ef-a6e5-7d1ff9569600 None None] Network e57f6e29-4637-455e-af9d-0975c45c995c requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:31:48.825426 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-593607'] for network, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:48.903786 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] Deleting port a6b8dc33-0ff3-46bf-a09b-b512222443bb {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:31:48.904508 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:49.018340 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] For port a6b8dc33-0ff3-46bf-a09b-b512222443bb, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=a6b8dc33-0ff3-46bf-a09b-b512222443bb,segment=NetworkSegment(c7d77442-5ac6-48cf-ab59-3f5962fc1e3e),segment_id=c7d77442-5ac6-48cf-ab59-3f5962fc1e3e)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:31:49.018715 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:49.022863 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-268bff29-47db-4160-9d34-d6a12100e2c3 req-db2a5e46-f66b-4b4e-b4e7-c2ba0b70e9f3 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:49.024502 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-268bff29-47db-4160-9d34-d6a12100e2c3 req-db2a5e46-f66b-4b4e-b4e7-c2ba0b70e9f3 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 943 time: 0.0942254 May 20 04:31:49.043585 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-f084ae44-6893-422a-9782-416efd5b2ef8 None None] DHCP port deletion for e57f6e29-4637-455e-af9d-0975c45c995c request from dv-cinder-782405-9-iscsi {{(pid=91645) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 20 04:31:49.083474 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-268bff29-47db-4160-9d34-d6a12100e2c3 req-fce9528c-788d-48ed-adf1-031179e527fa tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0526545 May 20 04:31:49.120697 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-f084ae44-6893-422a-9782-416efd5b2ef8 None None] Deleting port a6b8dc33-0ff3-46bf-a09b-b512222443bb {{(pid=91645) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:31:49.121078 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-f084ae44-6893-422a-9782-416efd5b2ef8 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:49.151636 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] Calling delete_port for a6b8dc33-0ff3-46bf-a09b-b512222443bb owned by network:dhcp {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:31:49.204522 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:49.210039 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:49.246617 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-f084ae44-6893-422a-9782-416efd5b2ef8 None None] For port a6b8dc33-0ff3-46bf-a09b-b512222443bb, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=a6b8dc33-0ff3-46bf-a09b-b512222443bb,segment=NetworkSegment(c7d77442-5ac6-48cf-ab59-3f5962fc1e3e),segment_id=c7d77442-5ac6-48cf-ab59-3f5962fc1e3e)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:31:49.247045 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-f084ae44-6893-422a-9782-416efd5b2ef8 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:49.257964 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] Pushing event deleted for resources: {'Port': ['ID=a6b8dc33-0ff3-46bf-a09b-b512222443bb,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:49.263121 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] No DVR routers for this DVR port a6b8dc33-0ff3-46bf-a09b-b512222443bb on host dv-cinder-782405-9-iscsi {{(pid=91644) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:31:49.264709 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:49.366453 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-f084ae44-6893-422a-9782-416efd5b2ef8 None None] Calling delete_port for a6b8dc33-0ff3-46bf-a09b-b512222443bb owned by network:dhcp {{(pid=91645) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:31:49.382563 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-7789108', 'neutron.services.segments.db._delete_segments_for_network-8760688958082'] for network, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:49.396016 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['e57f6e29-4637-455e-af9d-0975c45c995c']}} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:49.409319 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'c7d77442-5ac6-48cf-ab59-3f5962fc1e3e') {'for_net_delete': True} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:49.409667 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'c7d77442-5ac6-48cf-ab59-3f5962fc1e3e') {} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:49.424876 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8760688968331', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-853278'] for segment, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:49.441890 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:49.448543 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] Releasing vlan 806 on physical network physnet1 to pool {{(pid=91644) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 20 04:31:49.448921 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2001055'] for segment, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: Traceback (most recent call last): May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: timer() May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: cb(*args, **kw) May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: return func(*args, **kwargs) May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: self._notify() May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: self.callback(batched_events) May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: event.method(event) May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: aggregate_id = self._get_aggregate_id(event.segment_id) May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: aggregate_uuid = self.p_client.list_aggregates( May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: return f(self, *a, **k) May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: return self._get(url).json() May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: return self._client.get(url, endpoint_filter=self._ks_filter, May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: return self.request(url, 'GET', **kwargs) May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: base_url = self.get_endpoint(auth, allow=allow, May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: auth = self._auth_required(auth, 'determine endpoint URL') May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 20 04:31:49.462461 dv-cinder-782405-9-iscsi neutron-server[91644]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 20 04:31:49.517723 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-f084ae44-6893-422a-9782-416efd5b2ef8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716'] for port, after_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:49.522639 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:49.571345 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f084ae44-6893-422a-9782-416efd5b2ef8 None None] Pushing event deleted for resources: {'Port': ['ID=a6b8dc33-0ff3-46bf-a09b-b512222443bb,revision_number=None']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:49.573528 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f084ae44-6893-422a-9782-416efd5b2ef8 None None] No DVR routers for this DVR port a6b8dc33-0ff3-46bf-a09b-b512222443bb on host dv-cinder-782405-9-iscsi {{(pid=91645) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:31:49.575048 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-f084ae44-6893-422a-9782-416efd5b2ef8 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:49.661174 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-7789117', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-8053321'] for network, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:49.668786 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:49.672503 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e08c6e49-4b4f-4617-9409-469fe9ed7de1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:49.697344 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] Pushing event deleted for resources: {'Network': ['ID=e57f6e29-4637-455e-af9d-0975c45c995c,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:49.700842 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:49.705418 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-d79cc5f6-df5a-435b-a8ed-e3e4ac9994ce admin admin] 10.222.0.39 "DELETE /v2.0/networks/e57f6e29-4637-455e-af9d-0975c45c995c HTTP/1.1" status: 204 len: 168 time: 0.9956818 May 20 04:31:49.708745 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 42808) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:31:49.925030 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c5e149e1-98e9-4fad-887c-742bb8b78b3b req-45d9acc2-5593-4816-a979-c12f835bdab2 service neutron] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d&fields=id&fields=resource_request HTTP/1.1" status: 200 len: 255 time: 0.0760913 May 20 04:31:50.021016 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f3c309c7-c921-4650-a194-a30a22b7de15 req-bffc4ba5-b972-4e54-a480-b43fb9c51e86 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:50.028358 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f3c309c7-c921-4650-a194-a30a22b7de15 req-bffc4ba5-b972-4e54-a480-b43fb9c51e86 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.1484177 May 20 04:31:50.081599 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device a6b8dc33-0ff3-46bf-a09b-b512222443bb no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:31:50.097790 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f3c309c7-c921-4650-a194-a30a22b7de15 req-8596abe7-50ee-4df7-a702-229bdb02d5c8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0669477 May 20 04:31:50.118971 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: a6b8dc33-0ff3-46bf-a09b-b512222443bb {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:31:50.119801 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device a6b8dc33-0ff3-46bf-a09b-b512222443bb not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:31:50.120402 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 37393b04-8a64-4737-953b-1c02d4efce9f no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:31:50.156435 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: 37393b04-8a64-4737-953b-1c02d4efce9f {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:31:50.157089 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 37393b04-8a64-4737-953b-1c02d4efce9f not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:31:50.301594 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6d92f48f-f104-4f7f-9f18-6db4b7a48a95 req-ea5e8521-1ccd-4334-964f-75ee8a9e5910 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:50.303078 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6d92f48f-f104-4f7f-9f18-6db4b7a48a95 req-ea5e8521-1ccd-4334-964f-75ee8a9e5910 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 943 time: 0.0906854 May 20 04:31:50.304476 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a99a3227-7799-4f92-980a-32ef1d462ffd admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-8003145'] for router_interface, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:50.358007 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6d92f48f-f104-4f7f-9f18-6db4b7a48a95 req-1f590581-d88d-42f2-9034-9904e39c8a2d tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0472605 May 20 04:31:50.646689 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device fb2044c2-038a-473c-968f-29ae50d3c1a4 details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:31:50.649479 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b1d353b0-0c40-42fa-a24f-0e337c3c1d7c no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:31:50.721806 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-a99a3227-7799-4f92-980a-32ef1d462ffd admin admin] Deleting port 37ba7095-db27-4d59-9431-f8647c89ac1a {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:31:50.722204 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a99a3227-7799-4f92-980a-32ef1d462ffd admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:50.877303 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port fb2044c2-038a-473c-968f-29ae50d3c1a4, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=fb2044c2-038a-473c-968f-29ae50d3c1a4,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:31:50.883905 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [None req-a99a3227-7799-4f92-980a-32ef1d462ffd admin admin] For port 37ba7095-db27-4d59-9431-f8647c89ac1a, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=37ba7095-db27-4d59-9431-f8647c89ac1a,segment=NetworkSegment(6822d26b-245e-4501-a6f1-7f56846f560c),segment_id=6822d26b-245e-4501-a6f1-7f56846f560c)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:31:50.884260 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a99a3227-7799-4f92-980a-32ef1d462ffd admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:50.890848 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': 'fb2044c2-038a-473c-968f-29ae50d3c1a4', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'port_id': 'fb2044c2-038a-473c-968f-29ae50d3c1a4', 'mac_address': 'fa:16:3e:27:9b:b4', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 841, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.7'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:31:50.978124 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:50.994724 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id fb2044c2-038a-473c-968f-29ae50d3c1a4 {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:31:51.010646 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port fb2044c2-038a-473c-968f-29ae50d3c1a4, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=fb2044c2-038a-473c-968f-29ae50d3c1a4,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:31:51.012870 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-a99a3227-7799-4f92-980a-32ef1d462ffd admin admin] Calling delete_port for 37ba7095-db27-4d59-9431-f8647c89ac1a owned by network:router_interface {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:31:51.015757 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:51.017490 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:51.044667 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c5e149e1-98e9-4fad-887c-742bb8b78b3b req-f2a3f239-4699-4974-8c9b-714e7f88d19b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:51.046122 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c5e149e1-98e9-4fad-887c-742bb8b78b3b req-f2a3f239-4699-4974-8c9b-714e7f88d19b service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d&tenant_id=9fafdf3a29024ac588e5b24719735677 HTTP/1.1" status: 200 len: 1096 time: 0.2245951 May 20 04:31:51.081640 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=fb2044c2-038a-473c-968f-29ae50d3c1a4,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:51.084617 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.067s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:51.278852 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c5e149e1-98e9-4fad-887c-742bb8b78b3b req-c74b7dd2-07d1-480e-84b5-8d85e01ad1fa service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:51.280343 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c5e149e1-98e9-4fad-887c-742bb8b78b3b req-c74b7dd2-07d1-480e-84b5-8d85e01ad1fa service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=9fafdf3a29024ac588e5b24719735677&device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 1096 time: 0.2041500 May 20 04:31:51.285730 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6c644d00-e608-4916-a186-ae4874f32cbc req-d5b36843-ba2a-4b84-8738-7462176d287c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:51.287253 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6c644d00-e608-4916-a186-ae4874f32cbc req-d5b36843-ba2a-4b84-8738-7462176d287c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0868568 May 20 04:31:51.332313 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6c644d00-e608-4916-a186-ae4874f32cbc req-921a129c-dc9e-4d52-8104-94f7d01e5fb1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0386894 May 20 04:31:51.380110 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['fb2044c2-038a-473c-968f-29ae50d3c1a4'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:31:51.441629 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a99a3227-7799-4f92-980a-32ef1d462ffd admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:51.447644 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-0289fc2b-ec3c-4638-b5ff-7147661dd2c5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:51.502970 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a99a3227-7799-4f92-980a-32ef1d462ffd admin admin] Pushing event deleted for resources: {'Port': ['ID=37ba7095-db27-4d59-9431-f8647c89ac1a,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:51.507247 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-a99a3227-7799-4f92-980a-32ef1d462ffd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.060s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:51.532147 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c5e149e1-98e9-4fad-887c-742bb8b78b3b req-8f3e3f5d-15dc-4c5d-8dfd-c0c56bf1b7ad service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:51.533603 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c5e149e1-98e9-4fad-887c-742bb8b78b3b req-8f3e3f5d-15dc-4c5d-8dfd-c0c56bf1b7ad service neutron] 10.222.0.74 "GET /v2.0/networks?id=64bcea8a-6d35-406b-a823-fa1185008174 HTTP/1.1" status: 200 len: 918 time: 0.2462156 May 20 04:31:51.568641 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e0f228bb-e149-48c7-aaee-771e17cc9b74 req-cc40b734-9d9b-4f7d-88e7-9c6a285f1c45 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:51.570744 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e0f228bb-e149-48c7-aaee-771e17cc9b74 req-cc40b734-9d9b-4f7d-88e7-9c6a285f1c45 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 944 time: 0.0968487 May 20 04:31:51.629879 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e0f228bb-e149-48c7-aaee-771e17cc9b74 req-56f33abc-5ba8-4df2-9dd2-753004de6a4a tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0506294 May 20 04:31:51.840180 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c5e149e1-98e9-4fad-887c-742bb8b78b3b req-793ccae2-4862-49ae-a0b3-15e7c7d0e2ad service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c HTTP/1.1" status: 200 len: 998 time: 0.1850657 May 20 04:31:51.900446 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a99a3227-7799-4f92-980a-32ef1d462ffd admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-7924922', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-8054745'] for router_interface, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:51.970620 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c5e149e1-98e9-4fad-887c-742bb8b78b3b req-39264b1b-389e-4aa6-915a-29a81ff0eb7b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:51.972102 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c5e149e1-98e9-4fad-887c-742bb8b78b3b req-39264b1b-389e-4aa6-915a-29a81ff0eb7b service neutron] 10.222.0.74 "GET /v2.0/subnets?id=203a3962-2282-4f51-ac92-c9695b5c0cfe HTTP/1.1" status: 200 len: 856 time: 0.1240873 May 20 04:31:52.198286 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c5e149e1-98e9-4fad-887c-742bb8b78b3b req-fbbc07d7-718e-4385-9e9b-6dde880bfd78 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:52.199658 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c5e149e1-98e9-4fad-887c-742bb8b78b3b req-fbbc07d7-718e-4385-9e9b-6dde880bfd78 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=64bcea8a-6d35-406b-a823-fa1185008174&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.2196183 May 20 04:31:52.419863 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c5e149e1-98e9-4fad-887c-742bb8b78b3b req-b53017c2-5871-4b14-a88c-ff05e15cfc2d service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2126319 May 20 04:31:52.424036 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-a99a3227-7799-4f92-980a-32ef1d462ffd admin admin] Router ecb83b2f-735e-4418-8208-a3dfb9673052 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91644) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:31:52.526136 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e8f891f8-9f5f-4c32-8bb9-66673ffab38e req-ad2433c5-4027-4c66-bf1d-68b13fbe1300 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:52.527633 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e8f891f8-9f5f-4c32-8bb9-66673ffab38e req-ad2433c5-4027-4c66-bf1d-68b13fbe1300 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0853796 May 20 04:31:52.575020 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e8f891f8-9f5f-4c32-8bb9-66673ffab38e req-abf8195c-1f14-4b54-9c4e-29c89a6806ee tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0414672 May 20 04:31:52.668806 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device fb2044c2-038a-473c-968f-29ae50d3c1a4 up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:31:52.739198 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port fb2044c2-038a-473c-968f-29ae50d3c1a4 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:31:52.744317 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port fb2044c2-038a-473c-968f-29ae50d3c1a4 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:31:52.744698 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:52.821727 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-c5e149e1-98e9-4fad-887c-742bb8b78b3b req-270be0c5-6a67-4cea-abe6-0ba595d85824 service neutron] 10.222.0.74 "GET /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3943295 May 20 04:31:52.828645 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-67a4551c-1982-49a4-9eaa-da388a80e4fe req-7cefbfd1-29e6-41e0-a7db-72e0220490d1 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:52.830193 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-67a4551c-1982-49a4-9eaa-da388a80e4fe req-7cefbfd1-29e6-41e0-a7db-72e0220490d1 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 944 time: 0.0865264 May 20 04:31:52.832453 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-a99a3227-7799-4f92-980a-32ef1d462ffd admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:31:52.844398 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-499c5477-9a06-4c6c-acfb-ada0c90a0731 None None] Sync routers for ids ['ecb83b2f-735e-4418-8208-a3dfb9673052'] in dv-cinder-782405-9-iscsi {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:31:52.860425 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-a99a3227-7799-4f92-980a-32ef1d462ffd admin admin] 10.222.0.39 "PUT /v2.0/routers/ecb83b2f-735e-4418-8208-a3dfb9673052/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 3.1438525 May 20 04:31:52.860653 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-499c5477-9a06-4c6c-acfb-ada0c90a0731 None None] Routers {'ecb83b2f-735e-4418-8208-a3dfb9673052'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:31:52.861330 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-499c5477-9a06-4c6c-acfb-ada0c90a0731 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 9, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 9, "log_agent_heartbeats": false, "routers": 9},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:31:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['ecb83b2f-735e-4418-8208-a3dfb9673052'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:52.863272 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 42848) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:31:52.877045 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:52.886282 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-67a4551c-1982-49a4-9eaa-da388a80e4fe req-2b8d112b-8af0-48b8-98fd-afb5019123fb tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0492349 May 20 04:31:52.925418 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '7cc291a5-7b2b-48ad-af24-f0fbbbf92685', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'fb2044c2-038a-473c-968f-29ae50d3c1a4'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:31:52.936571 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port fb2044c2-038a-473c-968f-29ae50d3c1a4, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=fb2044c2-038a-473c-968f-29ae50d3c1a4,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:31:52.943074 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:52.947910 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:52.998419 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '7cc291a5-7b2b-48ad-af24-f0fbbbf92685', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'fb2044c2-038a-473c-968f-29ae50d3c1a4', 'code': 200} May 20 04:31:53.042882 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=fb2044c2-038a-473c-968f-29ae50d3c1a4,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:53.047144 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.101s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:53.082261 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-499c5477-9a06-4c6c-acfb-ada0c90a0731 None None] Set router ecb83b2f-735e-4418-8208-a3dfb9673052 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:31:53.122348 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-fe06bcfc-4713-4600-831c-23eb6f67c68c admin admin] Notify callbacks [] for router, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:53.364432 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-fe06bcfc-4713-4600-831c-23eb6f67c68c admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:53.379832 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1c396c88-d654-4827-8b78-41f30bedd19d req-6084c116-89a5-4715-94b1-86ca78cae810 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:53.381286 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-1c396c88-d654-4827-8b78-41f30bedd19d req-6084c116-89a5-4715-94b1-86ca78cae810 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 1098 time: 0.1207259 May 20 04:31:53.417167 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-499c5477-9a06-4c6c-acfb-ada0c90a0731 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'ecb83b2f-735e-4418-8208-a3dfb9673052', 'name': 'tempest-TestShelveInstance-2138875911-router', 'tenant_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.116'}], 'enable_snat': True}, 'gw_port_id': 'b57cd7ef-aeba-4bab-8870-543c319b914b', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:18Z', 'updated_at': '2021-05-20T04:31:51Z', 'revision_number': 5, 'project_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'gw_port': {'id': 'b57cd7ef-aeba-4bab-8870-543c319b914b', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:f5:44:9c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ecb83b2f-735e-4418-8208-a3dfb9673052', 'device_owner': 'network:router_gateway', 'standard_attr_id': 366, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.116', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:25:18Z', 'updated_at': '2021-05-20T04:25:24Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:53.421762 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-499c5477-9a06-4c6c-acfb-ada0c90a0731 None None] Checking router: ecb83b2f-735e-4418-8208-a3dfb9673052 for host: dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:31:53.437570 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-499c5477-9a06-4c6c-acfb-ada0c90a0731 None None] Routers {'ecb83b2f-735e-4418-8208-a3dfb9673052'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:31:53.438453 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-499c5477-9a06-4c6c-acfb-ada0c90a0731 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 9, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 9, "log_agent_heartbeats": false, "routers": 9},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:31:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['ecb83b2f-735e-4418-8208-a3dfb9673052'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:53.504391 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-fe06bcfc-4713-4600-831c-23eb6f67c68c admin admin] Deleting port b57cd7ef-aeba-4bab-8870-543c319b914b {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:31:53.504730 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-fe06bcfc-4713-4600-831c-23eb6f67c68c admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:53.625243 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-499c5477-9a06-4c6c-acfb-ada0c90a0731 None None] Set router ecb83b2f-735e-4418-8208-a3dfb9673052 gateway port host: None {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:31:53.685367 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [None req-fe06bcfc-4713-4600-831c-23eb6f67c68c admin admin] For port b57cd7ef-aeba-4bab-8870-543c319b914b, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=b57cd7ef-aeba-4bab-8870-543c319b914b,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:31:53.685749 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-fe06bcfc-4713-4600-831c-23eb6f67c68c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:53.768828 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-499c5477-9a06-4c6c-acfb-ada0c90a0731 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'ecb83b2f-735e-4418-8208-a3dfb9673052', 'name': 'tempest-TestShelveInstance-2138875911-router', 'tenant_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:18Z', 'updated_at': '2021-05-20T04:31:53Z', 'revision_number': 6, 'project_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'gw_port_host': None}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:53.773327 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-499c5477-9a06-4c6c-acfb-ada0c90a0731 None None] The sync data for ids ['ecb83b2f-735e-4418-8208-a3dfb9673052'] in dv-cinder-782405-9-iscsi is: [{'id': 'ecb83b2f-735e-4418-8208-a3dfb9673052', 'name': 'tempest-TestShelveInstance-2138875911-router', 'tenant_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:25:18Z', 'updated_at': '2021-05-20T04:31:53Z', 'revision_number': 6, 'project_id': '03fbb7d8cfaa4c2ab40ef558c109e377', 'gw_port_host': None}] {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:31:53.825864 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-500f3c89-6cf6-42c7-995e-0235c9dc41c7 req-87684a75-ddae-4c39-bd79-7a66406c2e88 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:53.827400 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-500f3c89-6cf6-42c7-995e-0235c9dc41c7 req-87684a75-ddae-4c39-bd79-7a66406c2e88 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0992832 May 20 04:31:53.836171 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-fe06bcfc-4713-4600-831c-23eb6f67c68c admin admin] Calling delete_port for b57cd7ef-aeba-4bab-8870-543c319b914b owned by network:router_gateway {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:31:53.892990 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-500f3c89-6cf6-42c7-995e-0235c9dc41c7 req-a8cdae85-8d93-42d8-9168-cdfd2ad03797 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0587988 May 20 04:31:54.066503 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 37ba7095-db27-4d59-9431-f8647c89ac1a no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:31:54.101724 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-fe06bcfc-4713-4600-831c-23eb6f67c68c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:54.109621 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: 37ba7095-db27-4d59-9431-f8647c89ac1a {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:31:54.109972 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 37ba7095-db27-4d59-9431-f8647c89ac1a not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:31:54.112768 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:54.183309 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fe06bcfc-4713-4600-831c-23eb6f67c68c admin admin] Pushing event deleted for resources: {'Port': ['ID=b57cd7ef-aeba-4bab-8870-543c319b914b,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:54.194111 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-fe06bcfc-4713-4600-831c-23eb6f67c68c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:54.255657 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-fe06bcfc-4713-4600-831c-23eb6f67c68c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-8053346'] for router_gateway, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:54.283777 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8fa86bdd-1bcb-4e6e-874e-85b095435a38 req-e8e6f95b-179e-4354-990d-edcd3421b6f3 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:54.287759 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8fa86bdd-1bcb-4e6e-874e-85b095435a38 req-e8e6f95b-179e-4354-990d-edcd3421b6f3 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.2014022 May 20 04:31:54.379487 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8fa86bdd-1bcb-4e6e-874e-85b095435a38 req-0b8c78ff-05e1-49bf-9b14-6c5894607650 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0873260 May 20 04:31:54.547625 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-fe06bcfc-4713-4600-831c-23eb6f67c68c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-8071641', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-260698'] for router, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:54.549857 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-fe06bcfc-4713-4600-831c-23eb6f67c68c admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:54.579500 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-fe06bcfc-4713-4600-831c-23eb6f67c68c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-8056629'] for router, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:54.580192 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-fe06bcfc-4713-4600-831c-23eb6f67c68c admin admin] Fanout notify agent at l3_agent the message router_deleted on router ecb83b2f-735e-4418-8208-a3dfb9673052 {{(pid=91644) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 20 04:31:54.594647 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-fe06bcfc-4713-4600-831c-23eb6f67c68c admin admin] Notify callbacks [] for router, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:54.598312 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-fe06bcfc-4713-4600-831c-23eb6f67c68c admin admin] 10.222.0.39 "DELETE /v2.0/routers/ecb83b2f-735e-4418-8208-a3dfb9673052 HTTP/1.1" status: 204 len: 168 time: 1.7310655 May 20 04:31:54.598596 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 42882) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:31:54.661454 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_v2 [None req-6c650f89-ad24-484f-ac51-2852ebf1e4d5 admin admin] Deleting subnet 747296cc-d241-444d-9f89-67006bc5546d {{(pid=91644) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 20 04:31:54.664650 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6c650f89-ad24-484f-ac51-2852ebf1e4d5 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:54.719038 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6c650f89-ad24-484f-ac51-2852ebf1e4d5 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-853257'] for subnet, precommit_delete_associations {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:54.802385 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6c650f89-ad24-484f-ac51-2852ebf1e4d5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-7786687'] for subnet, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:55.007233 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:31:55.065519 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6c650f89-ad24-484f-ac51-2852ebf1e4d5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-7786632', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-1988925', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-1810229'] for subnet, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:55.070497 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:55.102221 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-67b80435-cc38-4a8e-99c6-2b5124113041 req-f264f0c0-f0cc-4d58-9b1a-2bf4dea50cf2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:55.103720 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-67b80435-cc38-4a8e-99c6-2b5124113041 req-f264f0c0-f0cc-4d58-9b1a-2bf4dea50cf2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0866086 May 20 04:31:55.104074 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6c650f89-ad24-484f-ac51-2852ebf1e4d5 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=747296cc-d241-444d-9f89-67006bc5546d,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:55.107105 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-6c650f89-ad24-484f-ac51-2852ebf1e4d5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.037s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:55.108119 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6c650f89-ad24-484f-ac51-2852ebf1e4d5 admin admin] Notify callbacks [] for subnet, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:55.114218 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-6c650f89-ad24-484f-ac51-2852ebf1e4d5 admin admin] 10.222.0.39 "DELETE /v2.0/subnets/747296cc-d241-444d-9f89-67006bc5546d HTTP/1.1" status: 204 len: 168 time: 0.5147824 May 20 04:31:55.114378 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-3cbbe1a8-3231-4a6a-a6ad-8319ada9aae3 None None] Network c83ca109-2f8b-40c8-b2dd-b471ffdb582f requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:31:55.116912 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 42888) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:31:55.169469 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-67b80435-cc38-4a8e-99c6-2b5124113041 req-0c3e3547-1ac2-4d94-a9e1-520168fed751 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0452905 May 20 04:31:55.252041 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-593607'] for network, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:55.328580 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] Deleting port 424ccab3-7db4-445a-bbbc-4f034b243ce3 {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:31:55.328921 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:55.443826 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-fa328ca0-d2be-428f-98b5-af1ea68ab950 None None] DHCP port deletion for c83ca109-2f8b-40c8-b2dd-b471ffdb582f request from dv-cinder-782405-9-iscsi {{(pid=91645) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 20 04:31:55.445905 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] For port 424ccab3-7db4-445a-bbbc-4f034b243ce3, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=424ccab3-7db4-445a-bbbc-4f034b243ce3,segment=NetworkSegment(6822d26b-245e-4501-a6f1-7f56846f560c),segment_id=6822d26b-245e-4501-a6f1-7f56846f560c)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:31:55.446596 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:55.510407 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-fa328ca0-d2be-428f-98b5-af1ea68ab950 None None] Deleting port 424ccab3-7db4-445a-bbbc-4f034b243ce3 {{(pid=91645) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:31:55.510902 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-fa328ca0-d2be-428f-98b5-af1ea68ab950 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:55.582322 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] Calling delete_port for 424ccab3-7db4-445a-bbbc-4f034b243ce3 owned by network:dhcp {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:31:55.584553 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6552706-025b-491d-8617-10ec2384c73c req-10ef10cd-52ea-4537-bb42-2a75d25b6c5c tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:55.586263 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a6552706-025b-491d-8617-10ec2384c73c req-10ef10cd-52ea-4537-bb42-2a75d25b6c5c tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0989549 May 20 04:31:55.630023 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-fa328ca0-d2be-428f-98b5-af1ea68ab950 None None] For port 424ccab3-7db4-445a-bbbc-4f034b243ce3, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=424ccab3-7db4-445a-bbbc-4f034b243ce3,segment=NetworkSegment(6822d26b-245e-4501-a6f1-7f56846f560c),segment_id=6822d26b-245e-4501-a6f1-7f56846f560c)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:31:55.630418 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-fa328ca0-d2be-428f-98b5-af1ea68ab950 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:55.650449 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:55.655474 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-737f3f38-d99e-455f-b6dc-80b8cebde520 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:55.663837 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a6552706-025b-491d-8617-10ec2384c73c req-b0258b6f-562d-4746-9c5a-d39f46db6051 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0566697 May 20 04:31:55.706943 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] No DVR routers for this DVR port 424ccab3-7db4-445a-bbbc-4f034b243ce3 on host dv-cinder-782405-9-iscsi {{(pid=91644) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:31:55.708737 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] Pushing event deleted for resources: {'Port': ['ID=424ccab3-7db4-445a-bbbc-4f034b243ce3,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:55.736262 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:55.761077 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-fa328ca0-d2be-428f-98b5-af1ea68ab950 None None] Calling delete_port for 424ccab3-7db4-445a-bbbc-4f034b243ce3 owned by network:dhcp {{(pid=91645) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:31:55.807673 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-fa328ca0-d2be-428f-98b5-af1ea68ab950 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716'] for port, after_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:55.812140 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:55.829087 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-7789108', 'neutron.services.segments.db._delete_segments_for_network-8760688958082'] for network, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:55.843807 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['c83ca109-2f8b-40c8-b2dd-b471ffdb582f']}} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:55.857835 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '6822d26b-245e-4501-a6f1-7f56846f560c') {'for_net_delete': True} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:55.858120 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '6822d26b-245e-4501-a6f1-7f56846f560c') {} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:31:55.859018 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fa328ca0-d2be-428f-98b5-af1ea68ab950 None None] Pushing event deleted for resources: {'Port': ['ID=424ccab3-7db4-445a-bbbc-4f034b243ce3,revision_number=None']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:55.867596 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-fa328ca0-d2be-428f-98b5-af1ea68ab950 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:55.871897 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8760688968331', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-853278'] for segment, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:55.872160 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-fa328ca0-d2be-428f-98b5-af1ea68ab950 None None] No DVR routers for this DVR port 424ccab3-7db4-445a-bbbc-4f034b243ce3 on host dv-cinder-782405-9-iscsi {{(pid=91645) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:31:55.888824 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:55.895619 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] Releasing vlan 840 on physical network physnet1 to pool {{(pid=91644) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 20 04:31:55.896123 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2001055'] for segment, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: Traceback (most recent call last): May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: timer() May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: cb(*args, **kw) May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: return func(*args, **kwargs) May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: self._notify() May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: self.callback(batched_events) May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: event.method(event) May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: aggregate_id = self._get_aggregate_id(event.segment_id) May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: aggregate_uuid = self.p_client.list_aggregates( May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: return f(self, *a, **k) May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: return self._get(url).json() May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: return self._client.get(url, endpoint_filter=self._ks_filter, May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: return self.request(url, 'GET', **kwargs) May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: base_url = self.get_endpoint(auth, allow=allow, May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: auth = self._auth_required(auth, 'determine endpoint URL') May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 20 04:31:55.909134 dv-cinder-782405-9-iscsi neutron-server[91644]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 20 04:31:56.064412 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 424ccab3-7db4-445a-bbbc-4f034b243ce3 no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:31:56.089614 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-7789117', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-8053321'] for network, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:56.095901 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:56.099904 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e8f61247-4b3c-4382-93f9-76c7a05ca45a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:56.102372 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: 424ccab3-7db4-445a-bbbc-4f034b243ce3 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:31:56.102720 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 424ccab3-7db4-445a-bbbc-4f034b243ce3 not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:31:56.103148 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device b57cd7ef-aeba-4bab-8870-543c319b914b no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:31:56.121395 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] Pushing event deleted for resources: {'Network': ['ID=c83ca109-2f8b-40c8-b2dd-b471ffdb582f,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:56.124041 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.024s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:56.129034 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] 10.222.0.39 "DELETE /v2.0/networks/c83ca109-2f8b-40c8-b2dd-b471ffdb582f HTTP/1.1" status: 204 len: 168 time: 1.0111594 May 20 04:31:56.138053 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: b57cd7ef-aeba-4bab-8870-543c319b914b {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:31:56.138494 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device b57cd7ef-aeba-4bab-8870-543c319b914b not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:31:56.249818 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 42900) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:31:56.300199 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-cae579ed-dcab-4a54-b64f-b537b81f6696 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=39febf983bf24778a300f287dbeca829&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0491459 May 20 04:31:56.302660 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 42904) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:31:56.366735 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-cf5acb11-da9d-42a0-9e39-e53b80b94bfb admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:56.393571 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-cf5acb11-da9d-42a0-9e39-e53b80b94bfb admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:56.408247 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c2ecdc58-40dd-4fc0-8fbc-e32d61d51d2f req-645d7299-75d3-47e6-8820-9bfc0bf5b8c6 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:56.410609 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c2ecdc58-40dd-4fc0-8fbc-e32d61d51d2f req-645d7299-75d3-47e6-8820-9bfc0bf5b8c6 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0981467 May 20 04:31:56.458786 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-cf5acb11-da9d-42a0-9e39-e53b80b94bfb admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:56.463846 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-a0344952-c97a-4fd2-ae5a-4d32744d171f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:56.469786 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c2ecdc58-40dd-4fc0-8fbc-e32d61d51d2f req-d83e3a86-e3f2-4282-9bed-153bb50c5084 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0496149 May 20 04:31:56.476018 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cf5acb11-da9d-42a0-9e39-e53b80b94bfb admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=3e4fedd7-e290-4dd9-8e9b-c2652f05a367,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:56.479014 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-cf5acb11-da9d-42a0-9e39-e53b80b94bfb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:56.481009 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-cf5acb11-da9d-42a0-9e39-e53b80b94bfb admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:56.484995 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-cf5acb11-da9d-42a0-9e39-e53b80b94bfb admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/3e4fedd7-e290-4dd9-8e9b-c2652f05a367 HTTP/1.1" status: 204 len: 168 time: 0.1785684 May 20 04:31:56.837059 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 42914) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:31:56.850184 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9608aa2d-3f4d-480b-8031-e90bdd4b76c2 req-9811f4cf-721f-4b3c-ae82-4c20a9cf47ac tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:56.852018 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9608aa2d-3f4d-480b-8031-e90bdd4b76c2 req-9811f4cf-721f-4b3c-ae82-4c20a9cf47ac tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0838161 May 20 04:31:56.890417 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-fc0b36bd-e0c3-4498-b090-390c38aca468 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=03fbb7d8cfaa4c2ab40ef558c109e377&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0515192 May 20 04:31:56.892653 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 42918) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:31:56.910009 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9608aa2d-3f4d-480b-8031-e90bdd4b76c2 req-1662fa8d-ccea-4ea2-952d-a25d3c740293 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0497904 May 20 04:31:56.953355 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c2001ae1-8c48-4ab2-80ea-c7b69ccc9900 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:56.978226 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c2001ae1-8c48-4ab2-80ea-c7b69ccc9900 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:57.042141 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c2001ae1-8c48-4ab2-80ea-c7b69ccc9900 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:57.046172 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-f462a960-5514-4350-82cc-a09159d861fd admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:31:57.057287 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c2001ae1-8c48-4ab2-80ea-c7b69ccc9900 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e1a100be-6b01-4c74-80a6-a3bd788d8ca7,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:31:57.059952 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-c2001ae1-8c48-4ab2-80ea-c7b69ccc9900 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.014s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:31:57.071058 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c2001ae1-8c48-4ab2-80ea-c7b69ccc9900 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:31:57.072754 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-c2001ae1-8c48-4ab2-80ea-c7b69ccc9900 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/e1a100be-6b01-4c74-80a6-a3bd788d8ca7 HTTP/1.1" status: 204 len: 168 time: 0.1791947 May 20 04:31:57.668064 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-199577be-ffe6-4b8d-8991-2737a6ca4950 req-c151939c-0dc9-4f47-9149-7fadc8130689 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:57.669515 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-199577be-ffe6-4b8d-8991-2737a6ca4950 req-c151939c-0dc9-4f47-9149-7fadc8130689 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0815601 May 20 04:31:57.713873 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-199577be-ffe6-4b8d-8991-2737a6ca4950 req-0988b3f9-d34f-42ea-b170-eda4c96c9c18 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0381477 May 20 04:31:58.110713 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cc110265-480d-49c0-a22c-2bd2d92f562b req-70e7e069-75ff-49c8-8c0c-374599785755 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:58.112185 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cc110265-480d-49c0-a22c-2bd2d92f562b req-70e7e069-75ff-49c8-8c0c-374599785755 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0876417 May 20 04:31:58.156593 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cc110265-480d-49c0-a22c-2bd2d92f562b req-f0c7f61c-de6f-4524-91a6-927faa5dc7f8 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0386288 May 20 04:31:58.912374 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-71f5701b-9923-4d36-ac3d-6cc10bf17634 req-b79a3c81-84c6-4b6f-b3da-f578198a09f1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:58.913820 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-71f5701b-9923-4d36-ac3d-6cc10bf17634 req-b79a3c81-84c6-4b6f-b3da-f578198a09f1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0851595 May 20 04:31:58.967607 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-71f5701b-9923-4d36-ac3d-6cc10bf17634 req-c2542491-3b99-424a-ab57-97412fbec508 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0446358 May 20 04:31:59.360038 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-262d43af-fdca-482d-8b9b-a5b4e5daf040 req-95c98ae7-5d4d-4a71-914c-fb51125c3fbe tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:31:59.361385 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-262d43af-fdca-482d-8b9b-a5b4e5daf040 req-95c98ae7-5d4d-4a71-914c-fb51125c3fbe tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0884936 May 20 04:31:59.410538 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-262d43af-fdca-482d-8b9b-a5b4e5daf040 req-a481ce31-6649-4a03-943f-5acbe415f762 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0433049 May 20 04:32:00.170677 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-56dc27ae-e7ef-49f2-b90e-fb78b3069bc1 req-a3a89212-d105-4789-9e24-95198e7a69d3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:00.172129 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-56dc27ae-e7ef-49f2-b90e-fb78b3069bc1 req-a3a89212-d105-4789-9e24-95198e7a69d3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0850494 May 20 04:32:00.215864 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-56dc27ae-e7ef-49f2-b90e-fb78b3069bc1 req-ebecd980-4a6e-4444-8a98-e7f1897ec88a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0379229 May 20 04:32:00.611639 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-df8e2d17-460e-4238-948d-a18e5d0f383d req-1ec1b921-cf45-44ec-b26c-183f4f90a296 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:00.613127 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-df8e2d17-460e-4238-948d-a18e5d0f383d req-1ec1b921-cf45-44ec-b26c-183f4f90a296 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0917964 May 20 04:32:00.665915 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-df8e2d17-460e-4238-948d-a18e5d0f383d req-6a78b1fd-fcbb-4ecf-8d32-977de0cccf86 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0458927 May 20 04:32:01.430710 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-754e8023-4a11-49cb-9110-be626190a160 req-4331f85a-1d3e-451b-8c63-70d4d3cb5bd7 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:01.432196 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-754e8023-4a11-49cb-9110-be626190a160 req-4331f85a-1d3e-451b-8c63-70d4d3cb5bd7 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0904491 May 20 04:32:01.477506 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-754e8023-4a11-49cb-9110-be626190a160 req-7ee790c0-5051-400a-b6bb-c8be3b2f221c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0395639 May 20 04:32:01.863807 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8fab4232-e923-408a-bde1-b229a7746450 req-65d1d358-98aa-4cce-80e7-7bf632ff4e46 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:01.865870 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8fab4232-e923-408a-bde1-b229a7746450 req-65d1d358-98aa-4cce-80e7-7bf632ff4e46 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0926533 May 20 04:32:01.912878 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8fab4232-e923-408a-bde1-b229a7746450 req-1116efe7-4575-4b2e-8ff1-ac587ed1eee9 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0396166 May 20 04:32:02.682345 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-14648999-81fa-4cc0-9e0c-46a039d221d9 req-76087b87-be46-4a15-86ff-03aa15505901 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:02.683803 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-14648999-81fa-4cc0-9e0c-46a039d221d9 req-76087b87-be46-4a15-86ff-03aa15505901 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0859156 May 20 04:32:02.730655 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-14648999-81fa-4cc0-9e0c-46a039d221d9 req-e8c07ca4-99da-4340-9e6e-e8dd821d98a3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0381181 May 20 04:32:03.114717 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29e529a4-517d-45ad-a421-0cbe7852b816 req-2a9ba4a1-07d6-448f-912b-f0f561a94088 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:03.116239 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-29e529a4-517d-45ad-a421-0cbe7852b816 req-2a9ba4a1-07d6-448f-912b-f0f561a94088 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0900273 May 20 04:32:03.160874 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-29e529a4-517d-45ad-a421-0cbe7852b816 req-be239227-57d1-4c32-ae48-7f887949b2fe tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0387743 May 20 04:32:03.952889 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-737ca447-8068-456b-a31a-1cfe849310f1 req-b6fa38b9-72c3-4622-bae2-e6d4bca49bf3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:03.954317 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-737ca447-8068-456b-a31a-1cfe849310f1 req-b6fa38b9-72c3-4622-bae2-e6d4bca49bf3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.1001370 May 20 04:32:04.004118 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-737ca447-8068-456b-a31a-1cfe849310f1 req-02e8cba3-1e37-4858-9ee4-17cd76f056e3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0438473 May 20 04:32:04.357563 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5e944936-426e-4d1c-960b-ab733e0fb9cc req-d939ee44-4f56-48b8-83eb-c116cacc2a3d tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:04.359023 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5e944936-426e-4d1c-960b-ab733e0fb9cc req-d939ee44-4f56-48b8-83eb-c116cacc2a3d tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0871251 May 20 04:32:04.406130 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5e944936-426e-4d1c-960b-ab733e0fb9cc req-57e2926f-e5c3-495a-a3d4-a9391ad8d507 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0412028 May 20 04:32:05.222046 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc0d021c-7262-4d48-a5da-85754b69d354 req-86cd7754-3a24-4fb9-b12b-ea13e9a22f40 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:05.226801 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-dc0d021c-7262-4d48-a5da-85754b69d354 req-86cd7754-3a24-4fb9-b12b-ea13e9a22f40 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.1056223 May 20 04:32:05.357763 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-dc0d021c-7262-4d48-a5da-85754b69d354 req-3addd33e-b56e-48ad-8e71-87983327a168 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1298752 May 20 04:32:05.604295 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3d298305-01f2-4b47-a4e3-288f253c4979 req-c0330122-6cec-405c-875f-5981f489a230 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:05.606246 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3d298305-01f2-4b47-a4e3-288f253c4979 req-c0330122-6cec-405c-875f-5981f489a230 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0874133 May 20 04:32:05.629641 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:05.654015 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3d298305-01f2-4b47-a4e3-288f253c4979 req-b6ce8406-239b-4fac-bf17-37996cbdf14a tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0418561 May 20 04:32:06.574445 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b1a1b3ee-e71d-4220-beef-e211108afb87 req-274822f8-e260-4db5-b823-c50a91ba1a34 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:06.576063 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b1a1b3ee-e71d-4220-beef-e211108afb87 req-274822f8-e260-4db5-b823-c50a91ba1a34 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0973632 May 20 04:32:06.627980 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b1a1b3ee-e71d-4220-beef-e211108afb87 req-ce9be96f-e19d-4e6b-a130-4a8091dcacc9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0456913 May 20 04:32:06.860137 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a1046acc-1eb8-4a81-9a36-d3080dc16092 req-b1bbc51b-bd12-4a8d-90be-d0191d3bec3e tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:06.861589 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a1046acc-1eb8-4a81-9a36-d3080dc16092 req-b1bbc51b-bd12-4a8d-90be-d0191d3bec3e tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0892003 May 20 04:32:06.908968 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a1046acc-1eb8-4a81-9a36-d3080dc16092 req-be25c079-74bd-4aa2-938a-a010f9d1e863 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0409794 May 20 04:32:07.858903 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e73a89c1-8e37-49c1-b62f-a6943f75fb61 req-d90f38b8-839d-412f-9ab3-71b03230b486 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:07.860366 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e73a89c1-8e37-49c1-b62f-a6943f75fb61 req-d90f38b8-839d-412f-9ab3-71b03230b486 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0925725 May 20 04:32:07.905910 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e73a89c1-8e37-49c1-b62f-a6943f75fb61 req-12566c3f-5bc0-49c1-8304-ec2e83770d52 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0397096 May 20 04:32:08.111754 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0cea776-8bdd-4f64-9927-308236f2f1c3 req-b0f2a88a-2143-4c77-948a-2ffa0dbe0165 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:08.113733 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d0cea776-8bdd-4f64-9927-308236f2f1c3 req-b0f2a88a-2143-4c77-948a-2ffa0dbe0165 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0954149 May 20 04:32:08.159230 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d0cea776-8bdd-4f64-9927-308236f2f1c3 req-bfd2be9a-805b-485e-976a-7876ef27b515 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0391653 May 20 04:32:09.107178 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-aa2a695f-6b01-439e-b617-4207e24aff1b req-c7ebf1ca-ad32-4ee9-8798-03fb91232313 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:09.108666 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-aa2a695f-6b01-439e-b617-4207e24aff1b req-c7ebf1ca-ad32-4ee9-8798-03fb91232313 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0900521 May 20 04:32:09.157094 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-aa2a695f-6b01-439e-b617-4207e24aff1b req-3f2b92db-4dae-44d5-92ac-4b0d0e52ee53 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0425651 May 20 04:32:09.351484 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ec8938c9-bc1e-4767-959d-095a0125ee70 req-55243176-7560-4adc-92f8-39b6117a3031 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:09.352829 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ec8938c9-bc1e-4767-959d-095a0125ee70 req-55243176-7560-4adc-92f8-39b6117a3031 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0864456 May 20 04:32:09.399366 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ec8938c9-bc1e-4767-959d-095a0125ee70 req-9a1c8836-7034-4516-9c2e-0242fd0f77f1 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0406854 May 20 04:32:10.375421 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f55ebdf0-771b-407c-8cc9-42b9708d480c req-81ff8584-7308-41dc-bb51-a308461c4490 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:10.376833 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f55ebdf0-771b-407c-8cc9-42b9708d480c req-81ff8584-7308-41dc-bb51-a308461c4490 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0983734 May 20 04:32:10.423985 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f55ebdf0-771b-407c-8cc9-42b9708d480c req-e6536780-9511-4f46-bbcf-794b1ad061bb tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0413051 May 20 04:32:10.616571 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5c9c1c16-9774-421b-a5e6-2149e7d584d1 req-ba5379c2-b0da-48a6-84be-d7ece924521c tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:10.618116 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5c9c1c16-9774-421b-a5e6-2149e7d584d1 req-ba5379c2-b0da-48a6-84be-d7ece924521c tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0961208 May 20 04:32:10.674691 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5c9c1c16-9774-421b-a5e6-2149e7d584d1 req-b81350da-1108-430a-827b-734a70b7d506 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0473042 May 20 04:32:11.631863 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5f77c873-3de8-4fb8-a069-9976102d826c req-74ba2dcc-db05-46d0-9a7c-ef54ec9cbaf1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:11.633298 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5f77c873-3de8-4fb8-a069-9976102d826c req-74ba2dcc-db05-46d0-9a7c-ef54ec9cbaf1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0872872 May 20 04:32:11.680053 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5f77c873-3de8-4fb8-a069-9976102d826c req-0428559d-1108-42fb-b60e-b2f860ae6dc4 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0383482 May 20 04:32:11.872907 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ccea181-9c42-4746-aa86-3ec30bcf806e req-5fe318ff-59c6-41ba-b3bb-9f56941b446c tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:11.874365 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0ccea181-9c42-4746-aa86-3ec30bcf806e req-5fe318ff-59c6-41ba-b3bb-9f56941b446c tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0861156 May 20 04:32:11.926893 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0ccea181-9c42-4746-aa86-3ec30bcf806e req-70fb7ce2-fdaa-4fcc-87fb-d85de02864af tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0442963 May 20 04:32:12.626340 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-590c14e0-9623-40a9-b805-11d89a83339a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:12.895836 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-60dd55a7-8cd1-4f99-a04d-49ed190172c5 req-376f5ea6-9013-4468-ba9e-61c59d5a030c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:12.897208 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-60dd55a7-8cd1-4f99-a04d-49ed190172c5 req-376f5ea6-9013-4468-ba9e-61c59d5a030c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0989046 May 20 04:32:12.942489 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-60dd55a7-8cd1-4f99-a04d-49ed190172c5 req-536b0d47-332c-4d98-8c8a-886c5d8bb079 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0394297 May 20 04:32:13.141088 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c5d159ea-94eb-4a99-bb37-b6507bc70757 req-a0ec8cfc-7231-41b5-a230-de61e41b11e2 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:13.143094 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c5d159ea-94eb-4a99-bb37-b6507bc70757 req-a0ec8cfc-7231-41b5-a230-de61e41b11e2 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0945208 May 20 04:32:13.188527 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c5d159ea-94eb-4a99-bb37-b6507bc70757 req-3ad6b9f1-83c0-4e97-9175-61847a7195f8 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0392008 May 20 04:32:14.151749 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c3341cdd-b80d-4628-a9f3-3c5fd6946d2c req-2310d02c-011d-4d3a-ae1a-3c2a27435a24 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:14.153192 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c3341cdd-b80d-4628-a9f3-3c5fd6946d2c req-2310d02c-011d-4d3a-ae1a-3c2a27435a24 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0824931 May 20 04:32:14.197689 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c3341cdd-b80d-4628-a9f3-3c5fd6946d2c req-e1600c0d-a03e-4372-8914-d9d8a62a698d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0386832 May 20 04:32:14.394710 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9b1ec9e5-4c30-46f6-af88-647ac950eff5 req-19a98fc6-62ee-477b-ad79-82ffd19be9d5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:14.396167 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9b1ec9e5-4c30-46f6-af88-647ac950eff5 req-19a98fc6-62ee-477b-ad79-82ffd19be9d5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0933523 May 20 04:32:14.442293 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9b1ec9e5-4c30-46f6-af88-647ac950eff5 req-4bc883f7-0b69-4c1a-9849-b5bf3ec75ed6 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0403121 May 20 04:32:14.990310 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:15.112094 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-2d23a872-3d52-4600-8684-f5ef1e17cf87 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:15.406309 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-392814ce-7272-4758-ade2-9b0c27de413c req-ac1c73b3-fc81-459e-a28e-22c61863b916 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:15.407783 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-392814ce-7272-4758-ade2-9b0c27de413c req-ac1c73b3-fc81-459e-a28e-22c61863b916 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0894282 May 20 04:32:15.455572 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-392814ce-7272-4758-ade2-9b0c27de413c req-599297bd-fc89-4a03-bca5-6c729d3b6b06 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0419374 May 20 04:32:15.649025 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7d91248-e219-44b9-b208-10c8f226e55a req-9f628163-e30d-48d3-bb83-be6776b6a31c tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:15.650637 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a7d91248-e219-44b9-b208-10c8f226e55a req-9f628163-e30d-48d3-bb83-be6776b6a31c tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0961299 May 20 04:32:15.699099 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a7d91248-e219-44b9-b208-10c8f226e55a req-5a8b7ed1-2231-4200-90ec-5b60619dd4c7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0400035 May 20 04:32:16.228745 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:16.675431 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-291fc2af-43ee-4c9d-8de0-38b802c3adfb req-d8132308-201f-4503-b4d0-98144602f9cc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:16.676883 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-291fc2af-43ee-4c9d-8de0-38b802c3adfb req-d8132308-201f-4503-b4d0-98144602f9cc tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0970480 May 20 04:32:16.796767 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-291fc2af-43ee-4c9d-8de0-38b802c3adfb req-30ddaafe-e4e0-4fd4-9550-58e8a20cfd2a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1139443 May 20 04:32:16.978040 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2b35d079-c021-4b22-9268-9e92d4e2d257 req-bb48d4f6-38c6-411f-b629-affc18bcc8ba tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:16.979486 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b35d079-c021-4b22-9268-9e92d4e2d257 req-bb48d4f6-38c6-411f-b629-affc18bcc8ba tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0847383 May 20 04:32:17.029465 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b35d079-c021-4b22-9268-9e92d4e2d257 req-95c38503-e9d9-4904-893f-cd67a5e8f8b5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0412955 May 20 04:32:17.859814 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:18.003446 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b89003ee-8849-4e5c-9f6e-3da66874c077 req-eb89fb03-18b2-4729-acc9-102ead6d8d91 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:18.004848 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b89003ee-8849-4e5c-9f6e-3da66874c077 req-eb89fb03-18b2-4729-acc9-102ead6d8d91 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0919905 May 20 04:32:18.050619 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b89003ee-8849-4e5c-9f6e-3da66874c077 req-ed9ee321-f6cb-44cd-81d5-8987923504f6 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0380476 May 20 04:32:18.252178 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4c5ab1f8-22fe-4492-ba78-9399fc16ed19 req-2c8da7b2-2914-4a9a-9b6c-e450fbcaf6b0 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:18.253609 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4c5ab1f8-22fe-4492-ba78-9399fc16ed19 req-2c8da7b2-2914-4a9a-9b6c-e450fbcaf6b0 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.1004653 May 20 04:32:18.298349 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-4c5ab1f8-22fe-4492-ba78-9399fc16ed19 req-c39135cb-1f7a-44aa-aad0-15a4992e35ee tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0387788 May 20 04:32:19.254055 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fc104501-5401-40b9-b34c-bd97d9d31548 req-5769fb66-5af8-4c65-bdbc-8b270d941c38 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:19.255511 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fc104501-5401-40b9-b34c-bd97d9d31548 req-5769fb66-5af8-4c65-bdbc-8b270d941c38 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0880363 May 20 04:32:19.304502 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fc104501-5401-40b9-b34c-bd97d9d31548 req-35acaa32-cb53-4f88-9735-84227597e93c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0402496 May 20 04:32:19.501862 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-91f13951-5be7-4904-93a5-8f50a45fb4a0 req-c0954e51-1251-4def-9a4c-0ab4cc0dde39 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:19.503342 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-91f13951-5be7-4904-93a5-8f50a45fb4a0 req-c0954e51-1251-4def-9a4c-0ab4cc0dde39 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0918870 May 20 04:32:19.550245 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-91f13951-5be7-4904-93a5-8f50a45fb4a0 req-e1599ab2-5ce0-47b1-b35b-a7e3ecf4a97a tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0411229 May 20 04:32:20.505227 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e0d893dc-1ce1-4381-8706-2f78b64eaa0a req-8690c31f-c826-492a-8fef-9789b968c293 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:20.506843 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e0d893dc-1ce1-4381-8706-2f78b64eaa0a req-8690c31f-c826-492a-8fef-9789b968c293 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0826735 May 20 04:32:20.553749 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e0d893dc-1ce1-4381-8706-2f78b64eaa0a req-320e1331-8fb1-460a-b71d-dd1f9bbf8306 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0410769 May 20 04:32:20.752480 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9981ebab-1196-4303-8b13-d719458f677b req-c9653087-6bca-4e39-9f90-ea9524fd2b46 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:20.753906 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9981ebab-1196-4303-8b13-d719458f677b req-c9653087-6bca-4e39-9f90-ea9524fd2b46 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0910225 May 20 04:32:20.799824 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9981ebab-1196-4303-8b13-d719458f677b req-5b9ff8cd-b5d0-44d2-ad2a-bb66ecdf3041 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0399802 May 20 04:32:21.726317 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-1c396c88-d654-4827-8b78-41f30bedd19d req-a31b3e72-25bf-4a2b-ba47-a04b84bf932a service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.2047589 May 20 04:32:21.835728 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7af97478-67cd-4ce5-af7f-c5ce4d6e47aa req-6c1c2eb1-e438-4612-a512-1559948d5672 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:21.837739 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7af97478-67cd-4ce5-af7f-c5ce4d6e47aa req-6c1c2eb1-e438-4612-a512-1559948d5672 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.1636672 May 20 04:32:21.841557 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-63f9c4e0-38aa-4b73-8477-f8dea7e83f9e req-ed63b115-9659-49ff-a3de-d5f5d411e43e service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.2232020 May 20 04:32:21.885755 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7af97478-67cd-4ce5-af7f-c5ce4d6e47aa req-c31c3c6d-2dbf-4e32-a5c3-066cb581487c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0418582 May 20 04:32:22.001478 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ad72cb84-9bed-4687-af68-a7ed57523ced req-d9399848-c1aa-44eb-9b13-44d6f0280f3b tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:22.003009 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ad72cb84-9bed-4687-af68-a7ed57523ced req-d9399848-c1aa-44eb-9b13-44d6f0280f3b tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0920880 May 20 04:32:22.098917 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-ad72cb84-9bed-4687-af68-a7ed57523ced req-755f1bf7-6407-4a3a-814a-69d98261d761 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0887225 May 20 04:32:22.118075 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-63f9c4e0-38aa-4b73-8477-f8dea7e83f9e req-b31d2327-cde4-40b4-a7a5-f0f2b12bfecf service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:22.120233 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-63f9c4e0-38aa-4b73-8477-f8dea7e83f9e req-b31d2327-cde4-40b4-a7a5-f0f2b12bfecf service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=0e3baa0ea24d42f3ab59e40845b350fd&device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 1098 time: 0.0968955 May 20 04:32:22.296602 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-63f9c4e0-38aa-4b73-8477-f8dea7e83f9e req-b1715eb0-4855-4491-a04c-7d80f5c58caa service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:22.298011 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-63f9c4e0-38aa-4b73-8477-f8dea7e83f9e req-b1715eb0-4855-4491-a04c-7d80f5c58caa service neutron] 10.222.0.74 "GET /v2.0/networks?id=14c537db-01aa-4360-a348-47f94c75e03b HTTP/1.1" status: 200 len: 911 time: 0.1422341 May 20 04:32:22.409362 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-63f9c4e0-38aa-4b73-8477-f8dea7e83f9e req-04077271-4ac4-41da-b2f6-a1a6af82da90 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=ca396585-cdb5-4a50-a4c9-d5954043208e HTTP/1.1" status: 200 len: 1001 time: 0.1014557 May 20 04:32:22.484892 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-63f9c4e0-38aa-4b73-8477-f8dea7e83f9e req-f40ebf31-c9ca-4724-af8c-ab35e33fae87 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:22.486274 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-63f9c4e0-38aa-4b73-8477-f8dea7e83f9e req-f40ebf31-c9ca-4724-af8c-ab35e33fae87 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=cdea475c-2929-4ae4-9271-87cc2fb9a378 HTTP/1.1" status: 200 len: 849 time: 0.0670753 May 20 04:32:22.578091 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-63f9c4e0-38aa-4b73-8477-f8dea7e83f9e req-86488780-e8fa-41d2-b7c8-306281c8e4fa service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:22.579566 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-63f9c4e0-38aa-4b73-8477-f8dea7e83f9e req-86488780-e8fa-41d2-b7c8-306281c8e4fa service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=14c537db-01aa-4360-a348-47f94c75e03b&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0846069 May 20 04:32:22.592073 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [req-63f9c4e0-38aa-4b73-8477-f8dea7e83f9e req-4092078b-de13-4974-acd5-cbf3eaf933f9 service neutron] http://10.222.0.39:9696/v2.0/extensions returned with HTTP 200 {{(pid=91643) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 20 04:32:22.593338 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-63f9c4e0-38aa-4b73-8477-f8dea7e83f9e req-4092078b-de13-4974-acd5-cbf3eaf933f9 service neutron] 10.222.0.74 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0054615 May 20 04:32:22.726882 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-63f9c4e0-38aa-4b73-8477-f8dea7e83f9e req-0e0cbaa7-0076-4b0a-9404-fdc4de02eeab service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1253567 May 20 04:32:22.935040 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-63f9c4e0-38aa-4b73-8477-f8dea7e83f9e req-a4d203ce-a2f2-4565-b510-6200b076e3c2 service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2013726 May 20 04:32:23.091165 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cf3311d3-3c7d-4ca0-8be9-ea3789d53466 req-c08bc40e-5e43-4520-b8c9-84598b347c06 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:23.092964 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cf3311d3-3c7d-4ca0-8be9-ea3789d53466 req-c08bc40e-5e43-4520-b8c9-84598b347c06 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0935133 May 20 04:32:23.139417 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cf3311d3-3c7d-4ca0-8be9-ea3789d53466 req-4b8ed540-0902-4489-a1b3-84cb1895cdcd tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0403564 May 20 04:32:23.293978 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6027ff92-c8d6-4b0e-8f76-ccbda9055813 req-0587465f-5acc-4dd1-8969-06fd167261eb tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:23.295457 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6027ff92-c8d6-4b0e-8f76-ccbda9055813 req-0587465f-5acc-4dd1-8969-06fd167261eb tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0825157 May 20 04:32:23.342748 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-6027ff92-c8d6-4b0e-8f76-ccbda9055813 req-250fc43c-5162-4804-8c34-824e2e885ab4 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0410109 May 20 04:32:24.355706 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f87a60f3-91b5-45b3-b842-4915e868d860 req-d5317f03-e084-44b9-b83b-67062811a372 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:24.357050 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f87a60f3-91b5-45b3-b842-4915e868d860 req-d5317f03-e084-44b9-b83b-67062811a372 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 943 time: 0.0956545 May 20 04:32:24.405916 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f87a60f3-91b5-45b3-b842-4915e868d860 req-d6dcd463-5816-436e-a075-8484027df562 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0420914 May 20 04:32:24.539607 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2bb90bb4-ffc5-4c0c-bf66-ac966dda9063 req-76971ac8-6f4d-43be-84ef-63cc3fe8f896 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:24.541133 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2bb90bb4-ffc5-4c0c-bf66-ac966dda9063 req-76971ac8-6f4d-43be-84ef-63cc3fe8f896 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0913668 May 20 04:32:24.590416 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2bb90bb4-ffc5-4c0c-bf66-ac966dda9063 req-21dbe09c-bd51-4d47-b10d-b2d0e00d92ca tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0411601 May 20 04:32:24.818324 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5cee8884-f084-4fa1-8a61-1fd53cf01d85 req-bcbb6ae7-f29f-498c-893c-fd27194846e1 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:24.819789 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5cee8884-f084-4fa1-8a61-1fd53cf01d85 req-bcbb6ae7-f29f-498c-893c-fd27194846e1 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.1061585 May 20 04:32:24.832557 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b1d353b0-0c40-42fa-a24f-0e337c3c1d7c details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:32:24.866773 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5cee8884-f084-4fa1-8a61-1fd53cf01d85 req-b2e2b0bf-deaf-4b08-a002-64dde7ff1879 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0408745 May 20 04:32:24.890262 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43034) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:32:24.974996 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:32:24.992392 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'port_id': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c', 'mac_address': 'fa:16:3e:cf:31:63', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 847, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.9'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:32:25.074951 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-b07bb8a1-02e9-4e7f-ac1e-03af626a985c tempest-TestVolumeBootPattern-1668223768 tempest-TestVolumeBootPattern-1668223768-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:25.079170 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-b07bb8a1-02e9-4e7f-ac1e-03af626a985c tempest-TestVolumeBootPattern-1668223768 tempest-TestVolumeBootPattern-1668223768-project-admin] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 1093 time: 0.1858351 May 20 04:32:25.080194 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43042) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:32:25.080641 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:25.086146 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Request body: {'floatingip': {'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'port_id': 'fb2044c2-038a-473c-968f-29ae50d3c1a4', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'fixed_ip_address': '172.20.0.7'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:32:25.092643 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:32:25.096901 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Usage tracker for resource:floatingip and tenant:0e3baa0ea24d42f3ab59e40845b350fd is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:32:25.098529 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id b1d353b0-0c40-42fa-a24f-0e337c3c1d7c {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:32:25.100090 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Quota usage for floatingip was recalculated. Used quota:2. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:32:25.106209 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attempting to reserve 1 items for resource floatingip. Total usage: 2; quota limit: 50; headroom:48 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:32:25.114005 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:32:25.118083 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for floatingip, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:25.118799 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:25.121495 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:32:25.193073 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,revision_number=9']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:32:25.200265 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:32:25.200604 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:32:25.200935 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:32:25.201260 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:32:25.201583 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:32:25.201903 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:32:25.202218 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:32:25.202574 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:32:25.202998 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:32:25.203435 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:32:25.203941 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:32:25.204361 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:32:25.210211 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.089s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:32:25.259101 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:25.417683 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Allocated IP 172.27.0.163 (712ce8c4-8feb-4719-bf13-e06992125cd9/e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5/60843dfc-2df1-42ea-a3d7-059c71b23f9c) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:32:25.530364 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['b1d353b0-0c40-42fa-a24f-0e337c3c1d7c'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:32:25.568452 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:25.602762 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:25.608749 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7e8ec796-c50c-405f-8448-20465d7b6b8e req-c73083be-196b-47c1-b077-5c40bc8e1c56 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:25.610227 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7e8ec796-c50c-405f-8448-20465d7b6b8e req-c73083be-196b-47c1-b077-5c40bc8e1c56 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 944 time: 0.0846758 May 20 04:32:25.611983 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-acc60b92-fcfe-4e0d-9bca-d5b0e459f7a9 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:32:25.657446 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-7e8ec796-c50c-405f-8448-20465d7b6b8e req-c49bc20c-7a83-483e-9f68-8663bfb95bbb tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0408013 May 20 04:32:25.681589 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Network 712ce8c4-8feb-4719-bf13-e06992125cd9 is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:32:25.685715 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:32:25.690772 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:32:25.733482 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event updated for resources: {'Port': ['ID=60843dfc-2df1-42ea-a3d7-059c71b23f9c,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:32:25.741335 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.129s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:32:26.146647 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for floatingip, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:26.230363 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:26.547620 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:26.555735 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:26.559505 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:32:26.663939 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event updated for resources: {'Port': ['ID=60843dfc-2df1-42ea-a3d7-059c71b23f9c,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:32:26.667988 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.108s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:32:26.675689 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-8003251'] for floatingip, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:26.839644 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b1d353b0-0c40-42fa-a24f-0e337c3c1d7c up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:32:26.854809 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.db.l3_db [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Floating IP e1254a73-238b-496d-bda5-e60da4e8c2d4 associated. External IP: 172.27.0.163, port: fb2044c2-038a-473c-968f-29ae50d3c1a4. May 20 04:32:26.866709 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-50967068-f9f5-4af2-a2b3-02b8f3ae8cdb req-765c1a1e-de61-4bf9-8f38-ae37b8b96763 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:26.868349 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-50967068-f9f5-4af2-a2b3-02b8f3ae8cdb req-765c1a1e-de61-4bf9-8f38-ae37b8b96763 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 944 time: 0.0972624 May 20 04:32:26.917512 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-50967068-f9f5-4af2-a2b3-02b8f3ae8cdb req-d958bd41-6ab9-4483-8a7f-f05bb278d5e2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0433486 May 20 04:32:26.918378 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:32:26.918751 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:27.046701 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:27.073666 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '74d085a4-f817-48d6-b8d8-8504cc4ca99d', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:32:27.085746 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:32:27.090864 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:27.093673 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:32:27.132232 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '74d085a4-f817-48d6-b8d8-8504cc4ca99d', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c', 'code': 200} May 20 04:32:27.167621 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,revision_number=10']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:32:27.170794 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:32:27.244535 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Router 4969b258-f606-4def-b666-09764c475749 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91644) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:32:27.389798 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5e43d6d8-da00-43d8-9417-5eebcf3d6332 req-3f46763a-408e-4910-9e3e-dbcb9d461548 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:27.391298 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5e43d6d8-da00-43d8-9417-5eebcf3d6332 req-3f46763a-408e-4910-9e3e-dbcb9d461548 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 1099 time: 0.0780644 May 20 04:32:27.426955 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:32:27.436164 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5e43d6d8-da00-43d8-9417-5eebcf3d6332 req-a247d607-ab5d-4f29-adf8-6a6e984ab2b1 service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0366588 May 20 04:32:27.436798 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1e2b7b27-e185-41a5-b2bc-128f87fb9d9b None None] Sync routers for ids ['4969b258-f606-4def-b666-09764c475749'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:32:27.448793 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1e2b7b27-e185-41a5-b2bc-128f87fb9d9b None None] Routers {'4969b258-f606-4def-b666-09764c475749'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:32:27.449470 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-1e2b7b27-e185-41a5-b2bc-128f87fb9d9b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:32:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['4969b258-f606-4def-b666-09764c475749'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:32:27.452098 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:27.508424 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 996 time: 2.4271009 May 20 04:32:27.509143 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '7cc291a5-7b2b-48ad-af24-f0fbbbf92685', 'tag': 'fb2044c2-038a-473c-968f-29ae50d3c1a4'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:32:27.642607 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-1e2b7b27-e185-41a5-b2bc-128f87fb9d9b None None] Set router 4969b258-f606-4def-b666-09764c475749 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:32:27.680088 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '7cc291a5-7b2b-48ad-af24-f0fbbbf92685', 'tag': 'fb2044c2-038a-473c-968f-29ae50d3c1a4', 'status': 'completed', 'code': 200} May 20 04:32:27.781540 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f4c94ac3-6b3e-47ef-ac2a-c1deef607ac7 req-984c97de-c088-4bc6-8bcf-e5903a981c0a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:27.782994 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f4c94ac3-6b3e-47ef-ac2a-c1deef607ac7 req-984c97de-c088-4bc6-8bcf-e5903a981c0a service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=0e3baa0ea24d42f3ab59e40845b350fd&device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 1098 time: 0.0729413 May 20 04:32:27.918910 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f4c94ac3-6b3e-47ef-ac2a-c1deef607ac7 req-21fce4bf-917b-4db6-bdbf-bbec47a5c171 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:27.920289 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f4c94ac3-6b3e-47ef-ac2a-c1deef607ac7 req-21fce4bf-917b-4db6-bdbf-bbec47a5c171 service neutron] 10.222.0.74 "GET /v2.0/networks?id=14c537db-01aa-4360-a348-47f94c75e03b HTTP/1.1" status: 200 len: 911 time: 0.1295950 May 20 04:32:27.991254 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-1e2b7b27-e185-41a5-b2bc-128f87fb9d9b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'e1254a73-238b-496d-bda5-e60da4e8c2d4', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'floating_ip_address': '172.27.0.163', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '4969b258-f606-4def-b666-09764c475749', 'port_id': 'fb2044c2-038a-473c-968f-29ae50d3c1a4', 'fixed_ip_address': '172.20.0.7', 'status': 'DOWN', 'standard_attr_id': 591, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'mac_address': 'fa:16:3e:27:9b:b4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7cc291a5-7b2b-48ad-af24-f0fbbbf92685', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:32:26Z', 'updated_at': '2021-05-20T04:32:26Z', 'revision_number': 0, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'fixed_ip_address_scope': None}, {'id': 'f7e7609b-a65e-451e-8db6-b6e1bcbaa2e9', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'floating_ip_address': '172.27.0.168', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '4969b258-f606-4def-b666-09764c475749', 'port_id': 'ca396585-cdb5-4a50-a4c9-d5954043208e', 'fixed_ip_address': '172.20.0.5', 'status': 'ACTIVE', 'standard_attr_id': 569, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'mac_address': 'fa:16:3e:37:08:e4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:29:15Z', 'updated_at': '2021-05-20T04:29:18Z', 'revision_number': 1, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:50Z', 'revision_number': 5, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:32:27.995454 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1e2b7b27-e185-41a5-b2bc-128f87fb9d9b None None] Checking router: 4969b258-f606-4def-b666-09764c475749 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:32:28.004479 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1e2b7b27-e185-41a5-b2bc-128f87fb9d9b None None] Routers {'4969b258-f606-4def-b666-09764c475749'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:32:28.005154 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-1e2b7b27-e185-41a5-b2bc-128f87fb9d9b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:32:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['4969b258-f606-4def-b666-09764c475749'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:32:28.030099 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f4c94ac3-6b3e-47ef-ac2a-c1deef607ac7 req-92004792-ca17-4f18-a837-d88ae0d8d897 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=fb2044c2-038a-473c-968f-29ae50d3c1a4 HTTP/1.1" status: 200 len: 999 time: 0.1015916 May 20 04:32:28.158343 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f4c94ac3-6b3e-47ef-ac2a-c1deef607ac7 req-1e08af27-97c1-4035-8bd7-194e0dcf104a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:28.160232 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f4c94ac3-6b3e-47ef-ac2a-c1deef607ac7 req-1e08af27-97c1-4035-8bd7-194e0dcf104a service neutron] 10.222.0.74 "GET /v2.0/subnets?id=cdea475c-2929-4ae4-9271-87cc2fb9a378 HTTP/1.1" status: 200 len: 849 time: 0.1208794 May 20 04:32:28.181684 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-1e2b7b27-e185-41a5-b2bc-128f87fb9d9b None None] Set router 4969b258-f606-4def-b666-09764c475749 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:32:28.222878 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-108ed430-2f3a-4052-a0fe-30ba3acc2b58 req-686d5966-1dbb-4f46-9f62-7f4ecd4036d9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:28.224326 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-108ed430-2f3a-4052-a0fe-30ba3acc2b58 req-686d5966-1dbb-4f46-9f62-7f4ecd4036d9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 946 time: 0.1863837 May 20 04:32:28.279875 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-108ed430-2f3a-4052-a0fe-30ba3acc2b58 req-724750a0-da10-4c93-ae90-f77a8d48f8c2 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0469339 May 20 04:32:28.288507 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f4c94ac3-6b3e-47ef-ac2a-c1deef607ac7 req-1122722a-12ca-4ceb-b10e-537cf84f7b98 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:28.289949 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f4c94ac3-6b3e-47ef-ac2a-c1deef607ac7 req-1122722a-12ca-4ceb-b10e-537cf84f7b98 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=14c537db-01aa-4360-a348-47f94c75e03b&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1213484 May 20 04:32:28.431296 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f4c94ac3-6b3e-47ef-ac2a-c1deef607ac7 req-d868c798-5c91-4a5f-8ce1-3f26dd66953c service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1336644 May 20 04:32:28.602525 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-1e2b7b27-e185-41a5-b2bc-128f87fb9d9b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'e1254a73-238b-496d-bda5-e60da4e8c2d4', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'floating_ip_address': '172.27.0.163', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '4969b258-f606-4def-b666-09764c475749', 'port_id': 'fb2044c2-038a-473c-968f-29ae50d3c1a4', 'fixed_ip_address': '172.20.0.7', 'status': 'DOWN', 'standard_attr_id': 591, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'mac_address': 'fa:16:3e:27:9b:b4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7cc291a5-7b2b-48ad-af24-f0fbbbf92685', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:32:26Z', 'updated_at': '2021-05-20T04:32:26Z', 'revision_number': 0, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'fixed_ip_address_scope': None}, {'id': 'f7e7609b-a65e-451e-8db6-b6e1bcbaa2e9', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'floating_ip_address': '172.27.0.168', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '4969b258-f606-4def-b666-09764c475749', 'port_id': 'ca396585-cdb5-4a50-a4c9-d5954043208e', 'fixed_ip_address': '172.20.0.5', 'status': 'ACTIVE', 'standard_attr_id': 569, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'mac_address': 'fa:16:3e:37:08:e4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:29:15Z', 'updated_at': '2021-05-20T04:29:18Z', 'revision_number': 1, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:50Z', 'revision_number': 5, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:32:28.607732 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1e2b7b27-e185-41a5-b2bc-128f87fb9d9b None None] The sync data for ids ['4969b258-f606-4def-b666-09764c475749'] in dv-cinder-782405-9-iscsi is: [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'e1254a73-238b-496d-bda5-e60da4e8c2d4', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'floating_ip_address': '172.27.0.163', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '4969b258-f606-4def-b666-09764c475749', 'port_id': 'fb2044c2-038a-473c-968f-29ae50d3c1a4', 'fixed_ip_address': '172.20.0.7', 'status': 'DOWN', 'standard_attr_id': 591, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'mac_address': 'fa:16:3e:27:9b:b4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7cc291a5-7b2b-48ad-af24-f0fbbbf92685', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:32:26Z', 'updated_at': '2021-05-20T04:32:26Z', 'revision_number': 0, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'fixed_ip_address_scope': None}, {'id': 'f7e7609b-a65e-451e-8db6-b6e1bcbaa2e9', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'floating_ip_address': '172.27.0.168', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '4969b258-f606-4def-b666-09764c475749', 'port_id': 'ca396585-cdb5-4a50-a4c9-d5954043208e', 'fixed_ip_address': '172.20.0.5', 'status': 'ACTIVE', 'standard_attr_id': 569, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'mac_address': 'fa:16:3e:37:08:e4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:29:15Z', 'updated_at': '2021-05-20T04:29:18Z', 'revision_number': 1, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:50Z', 'revision_number': 5, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:32:28.671696 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f4c94ac3-6b3e-47ef-ac2a-c1deef607ac7 req-79f8152e-da00-462b-8e34-f400f0e732c3 service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2232029 May 20 04:32:28.806139 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-02c7e1f5-f713-4d7d-bcd9-95a3aec698ac None None] New status for floating IP e1254a73-238b-496d-bda5-e60da4e8c2d4: ACTIVE {{(pid=91646) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 20 04:32:29.485700 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-11b75e24-fc21-4b8b-ab09-e67592d8783d req-b640a234-2e0f-4b0c-859c-e70900ebdcef tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:29.487439 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-11b75e24-fc21-4b8b-ab09-e67592d8783d req-b640a234-2e0f-4b0c-859c-e70900ebdcef tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 946 time: 0.0899456 May 20 04:32:29.535455 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-11b75e24-fc21-4b8b-ab09-e67592d8783d req-6e00e3f3-aea3-4c9f-80bf-e9e67cded5f6 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0421419 May 20 04:32:30.730906 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e63fbfff-a172-4db1-940f-f83f2eed3fb9 req-560a4ffd-f122-42fd-875b-c5e8a4bdc15a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:32:30.732365 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e63fbfff-a172-4db1-940f-f83f2eed3fb9 req-560a4ffd-f122-42fd-875b-c5e8a4bdc15a tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 946 time: 0.0818288 May 20 04:32:30.783335 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e63fbfff-a172-4db1-940f-f83f2eed3fb9 req-9bd2f5f6-549e-4bc3-a96d-38133cccc451 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0452754 May 20 04:32:32.011537 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:32:35.647023 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:42.632256 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-1eb8b377-3ff4-415a-ad85-3aea2d238769 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:44.984979 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:45.107005 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-d6eca240-7dcf-4adf-b373-840bef024897 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:46.227641 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:32:47.862935 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:04.935702 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f9ac897e-6f30-47e8-b5fd-991f8e3b57d6 req-799f6897-9055-431e-b3d1-f8588597ca0b tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-79860973&fields=id&tenant_id=13d0e12c764f483ea457246b10c4cf7e HTTP/1.1" status: 200 len: 267 time: 0.0413978 May 20 04:33:05.018984 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f9ac897e-6f30-47e8-b5fd-991f8e3b57d6 req-9675e08c-dbb6-400f-9d5c-619fe783c421 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:05.020425 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f9ac897e-6f30-47e8-b5fd-991f8e3b57d6 req-9675e08c-dbb6-400f-9d5c-619fe783c421 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 985 time: 0.0794926 May 20 04:33:05.031833 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [req-f9ac897e-6f30-47e8-b5fd-991f8e3b57d6 req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Request body: {'port': {'security_groups': ['f596c737-91de-4c7f-9aa1-e6e95d668a70']}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:33:05.158318 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-f9ac897e-6f30-47e8-b5fd-991f8e3b57d6 req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:05.403812 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-f9ac897e-6f30-47e8-b5fd-991f8e3b57d6 req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] For port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=5f77cfe6-85e2-4a40-9a27-6ed37313ae13,segment=NetworkSegment(d558b057-1c72-4efe-ab53-567b572e2de5),segment_id=d558b057-1c72-4efe-ab53-567b572e2de5)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:33:05.433431 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-f9ac897e-6f30-47e8-b5fd-991f8e3b57d6 req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for port, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:05.439352 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-f9ac897e-6f30-47e8-b5fd-991f8e3b57d6 req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2070533'] for port, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:05.450020 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-e6a8695d-f4a1-4838-84f5-f9c402b399fe tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:05.452075 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['5f77cfe6-85e2-4a40-9a27-6ed37313ae13'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:33:05.509069 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f9ac897e-6f30-47e8-b5fd-991f8e3b57d6 req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:05.509936 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-f9ac897e-6f30-47e8-b5fd-991f8e3b57d6 req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:05.515284 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f9ac897e-6f30-47e8-b5fd-991f8e3b57d6 req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "PUT /v2.0/ports/5f77cfe6-85e2-4a40-9a27-6ed37313ae13 HTTP/1.1" status: 200 len: 943 time: 0.4881990 May 20 04:33:05.517621 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43232) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:05.539472 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'dfc47d7d-eb29-4bb6-a680-660af78ffcf3', 'tag': '5f77cfe6-85e2-4a40-9a27-6ed37313ae13'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:33:05.604065 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Pushing event updated for resources: {'Port': ['ID=5f77cfe6-85e2-4a40-9a27-6ed37313ae13,revision_number=7']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:05.613815 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.614377 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.614916 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.615483 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.616017 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.616547 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.617082 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.617608 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.618169 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.618690 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.619226 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.619737 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.633566 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.185s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:05.636500 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-778a8260-046a-4610-b83c-64fb5b0e408a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:05.670242 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-778a8260-046a-4610-b83c-64fb5b0e408a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:05.679830 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['5f77cfe6-85e2-4a40-9a27-6ed37313ae13'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:33:05.737312 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'dfc47d7d-eb29-4bb6-a680-660af78ffcf3', 'tag': '5f77cfe6-85e2-4a40-9a27-6ed37313ae13', 'status': 'completed', 'code': 200} May 20 04:33:05.745613 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-778a8260-046a-4610-b83c-64fb5b0e408a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:05.756914 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-92549aa6-8527-4538-acee-14bb37dfc870 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:05.779722 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-778a8260-046a-4610-b83c-64fb5b0e408a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=3a2163a0-97a4-4285-8fe2-96f2f61334ec,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:05.782101 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-778a8260-046a-4610-b83c-64fb5b0e408a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:05.785780 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-778a8260-046a-4610-b83c-64fb5b0e408a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "DELETE /v2.0/security-groups/3a2163a0-97a4-4285-8fe2-96f2f61334ec HTTP/1.1" status: 204 len: 168 time: 0.2640982 May 20 04:33:05.785934 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:05.788628 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43240) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:05.846457 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-778a8260-046a-4610-b83c-64fb5b0e408a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.846821 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-778a8260-046a-4610-b83c-64fb5b0e408a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.847388 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-778a8260-046a-4610-b83c-64fb5b0e408a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.847922 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-778a8260-046a-4610-b83c-64fb5b0e408a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.848277 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-778a8260-046a-4610-b83c-64fb5b0e408a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.848629 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-778a8260-046a-4610-b83c-64fb5b0e408a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.849321 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-778a8260-046a-4610-b83c-64fb5b0e408a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.849719 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-778a8260-046a-4610-b83c-64fb5b0e408a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.850062 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-778a8260-046a-4610-b83c-64fb5b0e408a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.850566 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-778a8260-046a-4610-b83c-64fb5b0e408a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.851078 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-778a8260-046a-4610-b83c-64fb5b0e408a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.851448 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-778a8260-046a-4610-b83c-64fb5b0e408a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:05.852702 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55a63a9b-5ed5-472b-9e8d-926f96841eff req-0d11f569-4388-4229-8442-5343b9b3039a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:05.854754 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-55a63a9b-5ed5-472b-9e8d-926f96841eff req-0d11f569-4388-4229-8442-5343b9b3039a service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=13d0e12c764f483ea457246b10c4cf7e&device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 1099 time: 0.0918050 May 20 04:33:05.861041 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-778a8260-046a-4610-b83c-64fb5b0e408a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:05.913285 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-4a36f830-e2f4-4e20-ba32-8d4b20506104 None None] Provisioning complete for port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 triggered by entity DHCP. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:33:05.913665 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-4a36f830-e2f4-4e20-ba32-8d4b20506104 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:05.997764 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55a63a9b-5ed5-472b-9e8d-926f96841eff req-16d08cdc-5072-49d6-a020-4f1a1f3c7238 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:05.999227 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-55a63a9b-5ed5-472b-9e8d-926f96841eff req-16d08cdc-5072-49d6-a020-4f1a1f3c7238 service neutron] 10.222.0.74 "GET /v2.0/networks?id=9798d104-d4de-413f-b41d-fb9d7e8a818e HTTP/1.1" status: 200 len: 914 time: 0.1368980 May 20 04:33:06.069221 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-aebf0826-3b92-4fc4-9bd5-789bcff7aa00 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Deleting port 6fc31c9c-218e-4616-bda1-b00cba81706e {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:06.069847 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-aebf0826-3b92-4fc4-9bd5-789bcff7aa00 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:06.109795 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-55a63a9b-5ed5-472b-9e8d-926f96841eff req-0bbcf430-6658-415e-a1db-b7f3537d22af service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=5f77cfe6-85e2-4a40-9a27-6ed37313ae13 HTTP/1.1" status: 200 len: 1001 time: 0.1032596 May 20 04:33:06.177148 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55a63a9b-5ed5-472b-9e8d-926f96841eff req-6c930f39-eb61-41cc-8467-e81e2b76b131 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:06.178612 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-55a63a9b-5ed5-472b-9e8d-926f96841eff req-6c930f39-eb61-41cc-8467-e81e2b76b131 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=c5423164-3d24-4562-871f-ab03914420dc HTTP/1.1" status: 200 len: 852 time: 0.0616653 May 20 04:33:06.255238 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55a63a9b-5ed5-472b-9e8d-926f96841eff req-9be2c423-769a-4771-8a53-a632fa665bdb service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:06.256730 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-55a63a9b-5ed5-472b-9e8d-926f96841eff req-9be2c423-769a-4771-8a53-a632fa665bdb service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=9798d104-d4de-413f-b41d-fb9d7e8a818e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0711141 May 20 04:33:06.294813 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-aebf0826-3b92-4fc4-9bd5-789bcff7aa00 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:06.295091 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-aebf0826-3b92-4fc4-9bd5-789bcff7aa00 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:06.386211 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-55a63a9b-5ed5-472b-9e8d-926f96841eff req-7a41d40e-cb2a-456c-97a8-2115002337f3 service neutron] 10.222.0.74 "GET /v2.0/networks/9798d104-d4de-413f-b41d-fb9d7e8a818e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1221664 May 20 04:33:06.415818 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-aebf0826-3b92-4fc4-9bd5-789bcff7aa00 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Calling delete_port for 6fc31c9c-218e-4616-bda1-b00cba81706e owned by network:floatingip {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:06.517772 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-55a63a9b-5ed5-472b-9e8d-926f96841eff req-a592e2e6-bfae-4c2f-8cc2-5d1a8ecf629d service neutron] 10.222.0.74 "GET /v2.0/networks/9798d104-d4de-413f-b41d-fb9d7e8a818e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1236520 May 20 04:33:06.539613 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-aebf0826-3b92-4fc4-9bd5-789bcff7aa00 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:06.544422 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-200bce83-74d0-4177-95ca-0e48818a7f42 tempest-AttachVolumeNegativeTest-1658348596 tempest-AttachVolumeNegativeTest-1658348596-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:06.586591 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aebf0826-3b92-4fc4-9bd5-789bcff7aa00 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Pushing event deleted for resources: {'Port': ['ID=6fc31c9c-218e-4616-bda1-b00cba81706e,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:06.589544 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-aebf0826-3b92-4fc4-9bd5-789bcff7aa00 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:06.684243 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-aebf0826-3b92-4fc4-9bd5-789bcff7aa00 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks [] for floatingip, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:06.684612 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.db.l3_db [None req-aebf0826-3b92-4fc4-9bd5-789bcff7aa00 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Floating IP 096511d5-89fe-4bdf-b446-22275adf2d5a disassociated (deleted). External IP: 172.27.0.74, port: 5f77cfe6-85e2-4a40-9a27-6ed37313ae13. May 20 04:33:07.036679 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-aebf0826-3b92-4fc4-9bd5-789bcff7aa00 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Router 7b630b1e-4360-4f2e-9053-be7e631bfa3a has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91644) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:33:07.222817 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-aebf0826-3b92-4fc4-9bd5-789bcff7aa00 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:33:07.232471 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-61e7ae06-f689-4558-9322-5a5a2bbbbcd2 None None] Sync routers for ids ['7b630b1e-4360-4f2e-9053-be7e631bfa3a'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:33:07.243776 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-61e7ae06-f689-4558-9322-5a5a2bbbbcd2 None None] Routers {'7b630b1e-4360-4f2e-9053-be7e631bfa3a'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:33:07.244409 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-61e7ae06-f689-4558-9322-5a5a2bbbbcd2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 5, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:32:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['7b630b1e-4360-4f2e-9053-be7e631bfa3a'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:07.246066 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-aebf0826-3b92-4fc4-9bd5-789bcff7aa00 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:07.300646 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-aebf0826-3b92-4fc4-9bd5-789bcff7aa00 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "DELETE /v2.0/floatingips/096511d5-89fe-4bdf-b446-22275adf2d5a HTTP/1.1" status: 204 len: 168 time: 1.5110245 May 20 04:33:07.301588 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'dfc47d7d-eb29-4bb6-a680-660af78ffcf3', 'tag': '5f77cfe6-85e2-4a40-9a27-6ed37313ae13'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:33:07.355504 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'dfc47d7d-eb29-4bb6-a680-660af78ffcf3', 'tag': '5f77cfe6-85e2-4a40-9a27-6ed37313ae13', 'status': 'completed', 'code': 200} May 20 04:33:07.428003 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-61e7ae06-f689-4558-9322-5a5a2bbbbcd2 None None] Set router 7b630b1e-4360-4f2e-9053-be7e631bfa3a gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:33:07.549830 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5ea9a074-847f-460e-899a-cce19d3bc4b0 req-9e8f8758-a6b5-42d4-97f0-ac0c8813dbf6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:07.551558 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5ea9a074-847f-460e-899a-cce19d3bc4b0 req-9e8f8758-a6b5-42d4-97f0-ac0c8813dbf6 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=13d0e12c764f483ea457246b10c4cf7e&device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 1099 time: 0.1743114 May 20 04:33:07.700101 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5ea9a074-847f-460e-899a-cce19d3bc4b0 req-1cadd38c-f6b9-417e-b1a3-047f60d553c8 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:07.701576 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5ea9a074-847f-460e-899a-cce19d3bc4b0 req-1cadd38c-f6b9-417e-b1a3-047f60d553c8 service neutron] 10.222.0.74 "GET /v2.0/networks?id=9798d104-d4de-413f-b41d-fb9d7e8a818e HTTP/1.1" status: 200 len: 914 time: 0.1436791 May 20 04:33:07.781967 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5ea9a074-847f-460e-899a-cce19d3bc4b0 req-0badcb87-5cc6-4116-b843-e991b503b058 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=5f77cfe6-85e2-4a40-9a27-6ed37313ae13 HTTP/1.1" status: 200 len: 217 time: 0.0730159 May 20 04:33:07.792771 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-61e7ae06-f689-4558-9322-5a5a2bbbbcd2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'name': 'tempest-TestMinimumBasicScenario-22498061-router', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158'}], 'enable_snat': True}, 'gw_port_id': '160f1898-b408-446e-8492-c4b331489205', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:24Z', 'updated_at': '2021-05-20T04:28:28Z', 'revision_number': 4, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'gw_port': {'id': '160f1898-b408-446e-8492-c4b331489205', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:79:e7:e5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 544, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:25Z', 'updated_at': '2021-05-20T04:28:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'bce609e7-1912-4202-bc3c-9028a36c0adb', 'name': '', 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'mac_address': 'fa:16:3e:57:64:c1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_interface', 'standard_attr_id': 545, 'fixed_ips': [{'subnet_id': 'c5423164-3d24-4562-871f-ab03914420dc', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:27Z', 'updated_at': '2021-05-20T04:28:34Z', 'revision_number': 5, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'subnets': [{'id': 'c5423164-3d24-4562-871f-ab03914420dc', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:07.796972 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-61e7ae06-f689-4558-9322-5a5a2bbbbcd2 None None] Checking router: 7b630b1e-4360-4f2e-9053-be7e631bfa3a for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:33:07.805295 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-61e7ae06-f689-4558-9322-5a5a2bbbbcd2 None None] Routers {'7b630b1e-4360-4f2e-9053-be7e631bfa3a'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:33:07.805826 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-61e7ae06-f689-4558-9322-5a5a2bbbbcd2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 5, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:32:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['7b630b1e-4360-4f2e-9053-be7e631bfa3a'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:07.848146 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5ea9a074-847f-460e-899a-cce19d3bc4b0 req-742cadde-e779-47e6-a89c-3ae254fd41c9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:07.849557 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5ea9a074-847f-460e-899a-cce19d3bc4b0 req-742cadde-e779-47e6-a89c-3ae254fd41c9 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=c5423164-3d24-4562-871f-ab03914420dc HTTP/1.1" status: 200 len: 852 time: 0.0604546 May 20 04:33:07.923609 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5ea9a074-847f-460e-899a-cce19d3bc4b0 req-d7ae3f4b-2861-42a3-9a0e-9a325672e6c3 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:07.925071 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5ea9a074-847f-460e-899a-cce19d3bc4b0 req-d7ae3f4b-2861-42a3-9a0e-9a325672e6c3 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=9798d104-d4de-413f-b41d-fb9d7e8a818e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0692706 May 20 04:33:07.980509 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-61e7ae06-f689-4558-9322-5a5a2bbbbcd2 None None] Set router 7b630b1e-4360-4f2e-9053-be7e631bfa3a gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:33:08.056824 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5ea9a074-847f-460e-899a-cce19d3bc4b0 req-0d76a3ab-76fa-4c61-b28d-f7d97920b8d1 service neutron] 10.222.0.74 "GET /v2.0/networks/9798d104-d4de-413f-b41d-fb9d7e8a818e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1245196 May 20 04:33:08.187455 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5ea9a074-847f-460e-899a-cce19d3bc4b0 req-bd05ce1d-4d13-45f9-8399-ab9469baa253 service neutron] 10.222.0.74 "GET /v2.0/networks/9798d104-d4de-413f-b41d-fb9d7e8a818e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1242139 May 20 04:33:08.330079 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-61e7ae06-f689-4558-9322-5a5a2bbbbcd2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'name': 'tempest-TestMinimumBasicScenario-22498061-router', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158'}], 'enable_snat': True}, 'gw_port_id': '160f1898-b408-446e-8492-c4b331489205', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:24Z', 'updated_at': '2021-05-20T04:28:28Z', 'revision_number': 4, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'gw_port': {'id': '160f1898-b408-446e-8492-c4b331489205', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:79:e7:e5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 544, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:25Z', 'updated_at': '2021-05-20T04:28:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'bce609e7-1912-4202-bc3c-9028a36c0adb', 'name': '', 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'mac_address': 'fa:16:3e:57:64:c1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_interface', 'standard_attr_id': 545, 'fixed_ips': [{'subnet_id': 'c5423164-3d24-4562-871f-ab03914420dc', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:27Z', 'updated_at': '2021-05-20T04:28:34Z', 'revision_number': 5, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'subnets': [{'id': 'c5423164-3d24-4562-871f-ab03914420dc', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:08.334027 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-61e7ae06-f689-4558-9322-5a5a2bbbbcd2 None None] The sync data for ids ['7b630b1e-4360-4f2e-9053-be7e631bfa3a'] in dv-cinder-782405-9-iscsi is: [{'id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'name': 'tempest-TestMinimumBasicScenario-22498061-router', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158'}], 'enable_snat': True}, 'gw_port_id': '160f1898-b408-446e-8492-c4b331489205', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:24Z', 'updated_at': '2021-05-20T04:28:28Z', 'revision_number': 4, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'gw_port': {'id': '160f1898-b408-446e-8492-c4b331489205', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:79:e7:e5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 544, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:25Z', 'updated_at': '2021-05-20T04:28:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'bce609e7-1912-4202-bc3c-9028a36c0adb', 'name': '', 'network_id': '9798d104-d4de-413f-b41d-fb9d7e8a818e', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'mac_address': 'fa:16:3e:57:64:c1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_interface', 'standard_attr_id': 545, 'fixed_ips': [{'subnet_id': 'c5423164-3d24-4562-871f-ab03914420dc', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:27Z', 'updated_at': '2021-05-20T04:28:34Z', 'revision_number': 5, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'subnets': [{'id': 'c5423164-3d24-4562-871f-ab03914420dc', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:33:08.542639 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-daad991f-8602-489d-8efe-35022747a350 None None] New status for floating IP 096511d5-89fe-4bdf-b446-22275adf2d5a: DOWN {{(pid=91646) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 20 04:33:09.016548 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:33:12.616743 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-6781678d-38ff-4c72-98a9-a2736ee50d42 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:12.928851 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b893258-93aa-4675-b950-a73f9d162d5d req-19cb3f7a-6f7e-4a62-a518-6bcbaedcf196 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0737636 May 20 04:33:13.027738 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2b893258-93aa-4675-b950-a73f9d162d5d req-d0820bb0-0ed8-4dfa-b112-cc692ad9937c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:13.029293 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b893258-93aa-4675-b950-a73f9d162d5d req-d0820bb0-0ed8-4dfa-b112-cc692ad9937c service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=13d0e12c764f483ea457246b10c4cf7e&device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 1099 time: 0.0735748 May 20 04:33:13.183920 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2b893258-93aa-4675-b950-a73f9d162d5d req-7889e162-ffdc-434a-8d81-a62d3858786f service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:13.185236 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b893258-93aa-4675-b950-a73f9d162d5d req-7889e162-ffdc-434a-8d81-a62d3858786f service neutron] 10.222.0.74 "GET /v2.0/networks?id=9798d104-d4de-413f-b41d-fb9d7e8a818e HTTP/1.1" status: 200 len: 914 time: 0.1286976 May 20 04:33:13.257442 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b893258-93aa-4675-b950-a73f9d162d5d req-a0f7188e-64dd-4c35-b20e-3a48774dea46 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=5f77cfe6-85e2-4a40-9a27-6ed37313ae13 HTTP/1.1" status: 200 len: 217 time: 0.0636389 May 20 04:33:13.323693 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2b893258-93aa-4675-b950-a73f9d162d5d req-5bdabed6-0c5b-48d5-b960-922c129c7c5c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:13.325289 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b893258-93aa-4675-b950-a73f9d162d5d req-5bdabed6-0c5b-48d5-b960-922c129c7c5c service neutron] 10.222.0.74 "GET /v2.0/subnets?id=c5423164-3d24-4562-871f-ab03914420dc HTTP/1.1" status: 200 len: 852 time: 0.0594769 May 20 04:33:13.405129 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2b893258-93aa-4675-b950-a73f9d162d5d req-6690fd82-fc05-4826-8db0-ef911a1d0423 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:13.406600 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b893258-93aa-4675-b950-a73f9d162d5d req-6690fd82-fc05-4826-8db0-ef911a1d0423 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=9798d104-d4de-413f-b41d-fb9d7e8a818e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0740833 May 20 04:33:13.542491 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b893258-93aa-4675-b950-a73f9d162d5d req-0a293478-f08a-41f0-bef3-9744fd904952 service neutron] 10.222.0.74 "GET /v2.0/networks/9798d104-d4de-413f-b41d-fb9d7e8a818e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1264317 May 20 04:33:13.675301 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-2b893258-93aa-4675-b950-a73f9d162d5d req-f49986a4-a050-4433-bf0b-b1f7b3bf5184 service neutron] 10.222.0.74 "GET /v2.0/networks/9798d104-d4de-413f-b41d-fb9d7e8a818e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1254692 May 20 04:33:14.994187 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:15.113555 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-b0f24f45-306e-4744-926c-1e63d1aefbcc None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:16.225971 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:16.287971 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f6766262-6e4c-4605-9956-cc80531debcf req-ea84e8e1-a20d-42a5-a4a6-0528ad87d991 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:16.289434 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f6766262-6e4c-4605-9956-cc80531debcf req-ea84e8e1-a20d-42a5-a4a6-0528ad87d991 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 946 time: 0.0875623 May 20 04:33:16.333071 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f6766262-6e4c-4605-9956-cc80531debcf req-78d290d2-1d2d-41f7-8118-6f4bcde66062 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0377858 May 20 04:33:17.609936 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-52bd90e2-ca8a-48d6-8c6f-4b06abf534a0 req-b01c5006-870f-40ed-9e80-7c2252342fd7 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:17.611409 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-52bd90e2-ca8a-48d6-8c6f-4b06abf534a0 req-b01c5006-870f-40ed-9e80-7c2252342fd7 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 946 time: 0.1687040 May 20 04:33:17.657265 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-52bd90e2-ca8a-48d6-8c6f-4b06abf534a0 req-82b97d5e-5399-42fa-bc16-0994efb0ae2d tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0379450 May 20 04:33:17.866108 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:18.447797 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9388ab3a-1df6-4ed6-a5a9-e0c85b2d8b2a req-45bd943a-25b5-4367-88c5-930c2abba90c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:18.449270 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9388ab3a-1df6-4ed6-a5a9-e0c85b2d8b2a req-45bd943a-25b5-4367-88c5-930c2abba90c tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.74 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 946 time: 0.0820479 May 20 04:33:18.588992 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-9388ab3a-1df6-4ed6-a5a9-e0c85b2d8b2a req-bf4ea990-2fac-4b27-9942-558b38f25d08 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Deleting port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:18.589332 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-9388ab3a-1df6-4ed6-a5a9-e0c85b2d8b2a req-bf4ea990-2fac-4b27-9942-558b38f25d08 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:18.822911 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-9388ab3a-1df6-4ed6-a5a9-e0c85b2d8b2a req-bf4ea990-2fac-4b27-9942-558b38f25d08 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] For port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=5f77cfe6-85e2-4a40-9a27-6ed37313ae13,segment=NetworkSegment(d558b057-1c72-4efe-ab53-567b572e2de5),segment_id=d558b057-1c72-4efe-ab53-567b572e2de5)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:33:18.823676 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-9388ab3a-1df6-4ed6-a5a9-e0c85b2d8b2a req-bf4ea990-2fac-4b27-9942-558b38f25d08 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:18.854876 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3ff79e40-609c-44fb-a0d3-a93d2ed5e96d req-81ec571a-4318-42ed-8d5f-546becbb0ad7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:18.856354 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3ff79e40-609c-44fb-a0d3-a93d2ed5e96d req-81ec571a-4318-42ed-8d5f-546becbb0ad7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 946 time: 0.2334347 May 20 04:33:18.966553 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-9388ab3a-1df6-4ed6-a5a9-e0c85b2d8b2a req-bf4ea990-2fac-4b27-9942-558b38f25d08 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Calling delete_port for 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 owned by compute:nova {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:19.028919 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3ff79e40-609c-44fb-a0d3-a93d2ed5e96d req-85e65380-2ece-4739-9d35-cea32957f33c tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=3eab1df2-4d7d-45a9-bcb2-293bd898fcc1&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0702257 May 20 04:33:19.045113 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a68957bd-ee09-49a2-ad87-851123488e08 req-3e395a32-50e6-4203-b850-d494f4c28557 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:19.046581 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a68957bd-ee09-49a2-ad87-851123488e08 req-3e395a32-50e6-4203-b850-d494f4c28557 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/ports?device_id=dfc47d7d-eb29-4bb6-a680-660af78ffcf3 HTTP/1.1" status: 200 len: 946 time: 0.2792881 May 20 04:33:19.138498 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a68957bd-ee09-49a2-ad87-851123488e08 req-b7a59d51-d309-4155-b874-a0d7dd8c232a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.39 "GET /v2.0/security-groups?id=f596c737-91de-4c7f-9aa1-e6e95d668a70&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0858691 May 20 04:33:19.169576 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-9388ab3a-1df6-4ed6-a5a9-e0c85b2d8b2a req-bf4ea990-2fac-4b27-9942-558b38f25d08 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:19.176098 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-a0277c2c-97c0-444e-bb41-a42f60c4252b service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:19.185959 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['5f77cfe6-85e2-4a40-9a27-6ed37313ae13'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:33:19.222580 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bf4ea990-2fac-4b27-9942-558b38f25d08 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Pushing event deleted for resources: {'Port': ['ID=5f77cfe6-85e2-4a40-9a27-6ed37313ae13,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:19.249419 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-bf4ea990-2fac-4b27-9942-558b38f25d08 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.073s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:19.298486 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [req-9388ab3a-1df6-4ed6-a5a9-e0c85b2d8b2a req-bf4ea990-2fac-4b27-9942-558b38f25d08 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] No DVR routers for this DVR port 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 on host hv-cinder-78240 {{(pid=91643) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:33:19.301361 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-9388ab3a-1df6-4ed6-a5a9-e0c85b2d8b2a req-bf4ea990-2fac-4b27-9942-558b38f25d08 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:19.304146 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'dfc47d7d-eb29-4bb6-a680-660af78ffcf3', 'name': 'network-vif-deleted', 'tag': '5f77cfe6-85e2-4a40-9a27-6ed37313ae13'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:33:19.317444 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9388ab3a-1df6-4ed6-a5a9-e0c85b2d8b2a req-bf4ea990-2fac-4b27-9942-558b38f25d08 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] 10.222.0.74 "DELETE /v2.0/ports/5f77cfe6-85e2-4a40-9a27-6ed37313ae13 HTTP/1.1" status: 204 len: 173 time: 0.8619235 May 20 04:33:19.354239 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'dfc47d7d-eb29-4bb6-a680-660af78ffcf3', 'name': 'network-vif-deleted', 'tag': '5f77cfe6-85e2-4a40-9a27-6ed37313ae13', 'status': 'completed', 'code': 200} May 20 04:33:20.225542 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3eb60a39-f3c9-4e2f-ab3a-22e598a29ee5 req-72d79982-cb67-4bd2-b81e-2a77148ac2fb tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:20.227031 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3eb60a39-f3c9-4e2f-ab3a-22e598a29ee5 req-72d79982-cb67-4bd2-b81e-2a77148ac2fb tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 946 time: 0.0854309 May 20 04:33:20.275027 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-3eb60a39-f3c9-4e2f-ab3a-22e598a29ee5 req-5ca3ab46-f570-41ca-8573-a7cbbb036319 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=3eab1df2-4d7d-45a9-bcb2-293bd898fcc1&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0399137 May 20 04:33:20.813676 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43394) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:21.345004 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-00dce94c-e5f9-44c8-b3af-f6bb993e78bf req-6424b5b0-e717-41b9-a583-d5102229f26e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:21.346326 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-00dce94c-e5f9-44c8-b3af-f6bb993e78bf req-6424b5b0-e717-41b9-a583-d5102229f26e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.74 "GET /v2.0/ports?device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 946 time: 0.0848234 May 20 04:33:21.455678 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0eb3eb37-5033-47e7-ae2e-fbf8cfc08c22 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-8003145'] for router_interface, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:21.540214 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-00919d67-c1d5-4c52-bb4f-ccca4aa1f6de req-16c379f4-432e-4174-b2e7-c04f3cae317c tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:21.541656 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-00919d67-c1d5-4c52-bb4f-ccca4aa1f6de req-16c379f4-432e-4174-b2e7-c04f3cae317c tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 946 time: 0.1552725 May 20 04:33:21.575233 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-00dce94c-e5f9-44c8-b3af-f6bb993e78bf req-adade246-1699-46dc-8be2-7be83e659574 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Deleting port c33841f1-fd78-4db6-9fee-4a5597eac2a5 {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:21.575584 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-00dce94c-e5f9-44c8-b3af-f6bb993e78bf req-adade246-1699-46dc-8be2-7be83e659574 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:21.629257 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-00919d67-c1d5-4c52-bb4f-ccca4aa1f6de req-c023ac9b-21f8-4efa-af7a-7de9f9e2ccf2 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups?id=3eab1df2-4d7d-45a9-bcb2-293bd898fcc1&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0812395 May 20 04:33:21.728507 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-00dce94c-e5f9-44c8-b3af-f6bb993e78bf req-adade246-1699-46dc-8be2-7be83e659574 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] For port c33841f1-fd78-4db6-9fee-4a5597eac2a5, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=c33841f1-fd78-4db6-9fee-4a5597eac2a5,segment=NetworkSegment(b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7),segment_id=b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:33:21.728858 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-00dce94c-e5f9-44c8-b3af-f6bb993e78bf req-adade246-1699-46dc-8be2-7be83e659574 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:21.870334 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-0eb3eb37-5033-47e7-ae2e-fbf8cfc08c22 admin admin] Deleting port bce609e7-1912-4202-bc3c-9028a36c0adb {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:21.870973 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0eb3eb37-5033-47e7-ae2e-fbf8cfc08c22 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:21.989857 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [None req-0eb3eb37-5033-47e7-ae2e-fbf8cfc08c22 admin admin] For port bce609e7-1912-4202-bc3c-9028a36c0adb, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=bce609e7-1912-4202-bc3c-9028a36c0adb,segment=NetworkSegment(d558b057-1c72-4efe-ab53-567b572e2de5),segment_id=d558b057-1c72-4efe-ab53-567b572e2de5)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:33:21.990108 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0eb3eb37-5033-47e7-ae2e-fbf8cfc08c22 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:22.113100 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-0eb3eb37-5033-47e7-ae2e-fbf8cfc08c22 admin admin] Calling delete_port for bce609e7-1912-4202-bc3c-9028a36c0adb owned by network:router_interface {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:22.116787 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-00dce94c-e5f9-44c8-b3af-f6bb993e78bf req-adade246-1699-46dc-8be2-7be83e659574 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for floatingip, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:22.117376 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-00dce94c-e5f9-44c8-b3af-f6bb993e78bf req-adade246-1699-46dc-8be2-7be83e659574 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-8003251'] for floatingip, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:22.117772 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.l3_db [req-00dce94c-e5f9-44c8-b3af-f6bb993e78bf req-adade246-1699-46dc-8be2-7be83e659574 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Floating IP e600ccfa-c7c6-497d-b86a-0d2286ccdd73 disassociated. External IP: 172.27.0.61, port: c33841f1-fd78-4db6-9fee-4a5597eac2a5. May 20 04:33:22.118175 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-00dce94c-e5f9-44c8-b3af-f6bb993e78bf req-adade246-1699-46dc-8be2-7be83e659574 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Calling delete_port for c33841f1-fd78-4db6-9fee-4a5597eac2a5 owned by compute:nova {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:22.232077 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-00dce94c-e5f9-44c8-b3af-f6bb993e78bf req-adade246-1699-46dc-8be2-7be83e659574 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:22.239382 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:22.261347 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0eb3eb37-5033-47e7-ae2e-fbf8cfc08c22 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:22.264879 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:22.287193 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-adade246-1699-46dc-8be2-7be83e659574 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Pushing event deleted for resources: {'Port': ['ID=c33841f1-fd78-4db6-9fee-4a5597eac2a5,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:22.289953 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-adade246-1699-46dc-8be2-7be83e659574 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:22.308422 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0eb3eb37-5033-47e7-ae2e-fbf8cfc08c22 admin admin] Pushing event deleted for resources: {'Port': ['ID=bce609e7-1912-4202-bc3c-9028a36c0adb,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:22.311821 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-0eb3eb37-5033-47e7-ae2e-fbf8cfc08c22 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:22.481934 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [req-00dce94c-e5f9-44c8-b3af-f6bb993e78bf req-adade246-1699-46dc-8be2-7be83e659574 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Router 10aeccef-e762-4811-a411-a1c9817db442 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91643) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:33:22.571569 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0eb3eb37-5033-47e7-ae2e-fbf8cfc08c22 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-7924922', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-8054745'] for router_interface, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:22.672424 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-00dce94c-e5f9-44c8-b3af-f6bb993e78bf req-adade246-1699-46dc-8be2-7be83e659574 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:33:22.687113 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-004fc474-f766-4710-a37a-af27371cd377 None None] Sync routers for ids ['10aeccef-e762-4811-a411-a1c9817db442'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:33:22.700939 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-004fc474-f766-4710-a37a-af27371cd377 None None] Routers {'10aeccef-e762-4811-a411-a1c9817db442'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:33:22.701588 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-004fc474-f766-4710-a37a-af27371cd377 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:33:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['10aeccef-e762-4811-a411-a1c9817db442'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:22.747113 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [req-00dce94c-e5f9-44c8-b3af-f6bb993e78bf req-adade246-1699-46dc-8be2-7be83e659574 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] No DVR routers for this DVR port c33841f1-fd78-4db6-9fee-4a5597eac2a5 on host hv-cinder-78240 {{(pid=91643) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:33:22.749649 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-00dce94c-e5f9-44c8-b3af-f6bb993e78bf req-adade246-1699-46dc-8be2-7be83e659574 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:22.758925 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4', 'name': 'network-vif-deleted', 'tag': 'c33841f1-fd78-4db6-9fee-4a5597eac2a5'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:33:22.764145 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-00dce94c-e5f9-44c8-b3af-f6bb993e78bf req-adade246-1699-46dc-8be2-7be83e659574 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.74 "DELETE /v2.0/ports/c33841f1-fd78-4db6-9fee-4a5597eac2a5 HTTP/1.1" status: 204 len: 173 time: 1.4111030 May 20 04:33:22.835532 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-917e5635-68fd-49ee-bdd8-b4c71d5f276d req-ae21a17a-5625-42e9-8600-b9a37e33ba0e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Synchronizing usage tracker for tenant:198bc8ab083f4a729f32b8ef27a6ee77 on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:33:22.840861 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4', 'name': 'network-vif-deleted', 'tag': 'c33841f1-fd78-4db6-9fee-4a5597eac2a5', 'status': 'completed', 'code': 200} May 20 04:33:22.861605 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-917e5635-68fd-49ee-bdd8-b4c71d5f276d req-ae21a17a-5625-42e9-8600-b9a37e33ba0e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Unset dirty status for tenant:198bc8ab083f4a729f32b8ef27a6ee77 on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:33:22.862895 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-917e5635-68fd-49ee-bdd8-b4c71d5f276d req-ae21a17a-5625-42e9-8600-b9a37e33ba0e tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/ports?device_id=cd2f4b31-7d83-4da8-9523-b6b6a6cc6cf4 HTTP/1.1" status: 200 len: 210 time: 0.0926762 May 20 04:33:22.956337 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-004fc474-f766-4710-a37a-af27371cd377 None None] Set router 10aeccef-e762-4811-a411-a1c9817db442 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:33:23.094269 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-0eb3eb37-5033-47e7-ae2e-fbf8cfc08c22 admin admin] Router 7b630b1e-4360-4f2e-9053-be7e631bfa3a has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91644) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:33:23.172708 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device c33841f1-fd78-4db6-9fee-4a5597eac2a5 no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:33:23.208270 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: c33841f1-fd78-4db6-9fee-4a5597eac2a5 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:33:23.208621 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device c33841f1-fd78-4db6-9fee-4a5597eac2a5 not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:33:23.209061 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:33:23.249639 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:33:23.249990 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 5f77cfe6-85e2-4a40-9a27-6ed37313ae13 not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:33:23.272049 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-0eb3eb37-5033-47e7-ae2e-fbf8cfc08c22 admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:33:23.282628 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-de1f2b77-2c43-4fdd-9e2e-ae162c9bc657 None None] Sync routers for ids ['7b630b1e-4360-4f2e-9053-be7e631bfa3a'] in dv-cinder-782405-9-iscsi {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:33:23.284004 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-0eb3eb37-5033-47e7-ae2e-fbf8cfc08c22 admin admin] 10.222.0.39 "PUT /v2.0/routers/7b630b1e-4360-4f2e-9053-be7e631bfa3a/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.4692390 May 20 04:33:23.286563 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43406) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:23.300626 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-de1f2b77-2c43-4fdd-9e2e-ae162c9bc657 None None] Routers {'7b630b1e-4360-4f2e-9053-be7e631bfa3a'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:33:23.301977 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-de1f2b77-2c43-4fdd-9e2e-ae162c9bc657 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:33:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['7b630b1e-4360-4f2e-9053-be7e631bfa3a'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:23.345806 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-004fc474-f766-4710-a37a-af27371cd377 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33'}], 'enable_snat': True}, 'gw_port_id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:24:36Z', 'revision_number': 4, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port': {'id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:64:1f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_gateway', 'standard_attr_id': 295, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:31Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '7dc69379-dd61-4add-8e0a-4ef00a211a0a', 'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'mac_address': 'fa:16:3e:a4:49:d4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_interface', 'standard_attr_id': 307, 'fixed_ips': [{'subnet_id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:35Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 5, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'subnets': [{'id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:23.350397 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-004fc474-f766-4710-a37a-af27371cd377 None None] Checking router: 10aeccef-e762-4811-a411-a1c9817db442 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:33:23.359413 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-004fc474-f766-4710-a37a-af27371cd377 None None] Routers {'10aeccef-e762-4811-a411-a1c9817db442'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:33:23.360034 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-004fc474-f766-4710-a37a-af27371cd377 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:33:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['10aeccef-e762-4811-a411-a1c9817db442'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:23.467100 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-be0e0973-ebf8-48c1-b1fc-92cd412d56eb admin admin] Notify callbacks [] for router, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:23.482308 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-de1f2b77-2c43-4fdd-9e2e-ae162c9bc657 None None] Set router 7b630b1e-4360-4f2e-9053-be7e631bfa3a gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:33:23.560239 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-004fc474-f766-4710-a37a-af27371cd377 None None] Set router 10aeccef-e762-4811-a411-a1c9817db442 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:33:23.650307 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-be0e0973-ebf8-48c1-b1fc-92cd412d56eb admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:23.724312 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-be0e0973-ebf8-48c1-b1fc-92cd412d56eb admin admin] Deleting port 160f1898-b408-446e-8492-c4b331489205 {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:23.724704 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-be0e0973-ebf8-48c1-b1fc-92cd412d56eb admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:23.771615 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-de1f2b77-2c43-4fdd-9e2e-ae162c9bc657 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'name': 'tempest-TestMinimumBasicScenario-22498061-router', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158'}], 'enable_snat': True}, 'gw_port_id': '160f1898-b408-446e-8492-c4b331489205', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:24Z', 'updated_at': '2021-05-20T04:33:22Z', 'revision_number': 5, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'gw_port': {'id': '160f1898-b408-446e-8492-c4b331489205', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:79:e7:e5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 544, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.158', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:25Z', 'updated_at': '2021-05-20T04:28:30Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:23.776167 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-de1f2b77-2c43-4fdd-9e2e-ae162c9bc657 None None] Checking router: 7b630b1e-4360-4f2e-9053-be7e631bfa3a for host: dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:33:23.785352 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-de1f2b77-2c43-4fdd-9e2e-ae162c9bc657 None None] Routers {'7b630b1e-4360-4f2e-9053-be7e631bfa3a'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:33:23.785979 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-de1f2b77-2c43-4fdd-9e2e-ae162c9bc657 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:33:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['7b630b1e-4360-4f2e-9053-be7e631bfa3a'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:23.849658 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [None req-be0e0973-ebf8-48c1-b1fc-92cd412d56eb admin admin] For port 160f1898-b408-446e-8492-c4b331489205, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=160f1898-b408-446e-8492-c4b331489205,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:33:23.850067 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-be0e0973-ebf8-48c1-b1fc-92cd412d56eb admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:23.934237 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-004fc474-f766-4710-a37a-af27371cd377 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33'}], 'enable_snat': True}, 'gw_port_id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:24:36Z', 'revision_number': 4, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port': {'id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:64:1f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_gateway', 'standard_attr_id': 295, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:31Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '7dc69379-dd61-4add-8e0a-4ef00a211a0a', 'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'mac_address': 'fa:16:3e:a4:49:d4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_interface', 'standard_attr_id': 307, 'fixed_ips': [{'subnet_id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:35Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 5, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'subnets': [{'id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:23.939694 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-004fc474-f766-4710-a37a-af27371cd377 None None] The sync data for ids ['10aeccef-e762-4811-a411-a1c9817db442'] in dv-cinder-782405-9-iscsi is: [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33'}], 'enable_snat': True}, 'gw_port_id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:24:36Z', 'revision_number': 4, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port': {'id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:64:1f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_gateway', 'standard_attr_id': 295, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:31Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '7dc69379-dd61-4add-8e0a-4ef00a211a0a', 'name': '', 'network_id': '49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'mac_address': 'fa:16:3e:a4:49:d4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_interface', 'standard_attr_id': 307, 'fixed_ips': [{'subnet_id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:35Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 5, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'subnets': [{'id': '4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:33:23.943393 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-de1f2b77-2c43-4fdd-9e2e-ae162c9bc657 None None] Set router 7b630b1e-4360-4f2e-9053-be7e631bfa3a gateway port host: None {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:33:23.983908 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-be0e0973-ebf8-48c1-b1fc-92cd412d56eb admin admin] Calling delete_port for 160f1898-b408-446e-8492-c4b331489205 owned by network:router_gateway {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:24.002574 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 43414) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:24.065221 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-de1f2b77-2c43-4fdd-9e2e-ae162c9bc657 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'name': 'tempest-TestMinimumBasicScenario-22498061-router', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:24Z', 'updated_at': '2021-05-20T04:33:23Z', 'revision_number': 6, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'gw_port_host': None}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:24.078775 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-de1f2b77-2c43-4fdd-9e2e-ae162c9bc657 None None] The sync data for ids ['7b630b1e-4360-4f2e-9053-be7e631bfa3a'] in dv-cinder-782405-9-iscsi is: [{'id': '7b630b1e-4360-4f2e-9053-be7e631bfa3a', 'name': 'tempest-TestMinimumBasicScenario-22498061-router', 'tenant_id': '13d0e12c764f483ea457246b10c4cf7e', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:24Z', 'updated_at': '2021-05-20T04:33:23Z', 'revision_number': 6, 'project_id': '13d0e12c764f483ea457246b10c4cf7e', 'gw_port_host': None}] {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:33:24.099407 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-953c9854-4b78-4b59-b1b1-176cccc2c2b4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:24.140136 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-953c9854-4b78-4b59-b1b1-176cccc2c2b4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:24.199500 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-953c9854-4b78-4b59-b1b1-176cccc2c2b4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:24.203735 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-e5be7a3b-f21e-47ae-a06e-d05909b668f6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:24.220272 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-953c9854-4b78-4b59-b1b1-176cccc2c2b4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=3eab1df2-4d7d-45a9-bcb2-293bd898fcc1,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:24.223053 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-953c9854-4b78-4b59-b1b1-176cccc2c2b4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:24.232436 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-953c9854-4b78-4b59-b1b1-176cccc2c2b4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:24.236055 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-953c9854-4b78-4b59-b1b1-176cccc2c2b4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "DELETE /v2.0/security-groups/3eab1df2-4d7d-45a9-bcb2-293bd898fcc1 HTTP/1.1" status: 204 len: 168 time: 0.2297401 May 20 04:33:24.236378 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 43418) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:24.255006 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.api.v2.resource [None req-2884bc48-cd31-4731-b892-56bfea1d80c8 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] show failed (client error): The resource could not be found. May 20 04:33:24.259092 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-2884bc48-cd31-4731-b892-56bfea1d80c8 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "GET /v2.0/security-groups/3eab1df2-4d7d-45a9-bcb2-293bd898fcc1 HTTP/1.1" status: 404 len: 347 time: 0.0181015 May 20 04:33:24.259258 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-be0e0973-ebf8-48c1-b1fc-92cd412d56eb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:24.259728 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 43422) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:24.260716 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-44ea8d0e-a80a-456c-b4e0-ec66a5fae0af tempest-VolumesNegativeTest-687684860 tempest-VolumesNegativeTest-687684860-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:24.277606 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-13cd628b-c637-49c6-b9e8-8d5abc8f6d1d None None] New status for floating IP e600ccfa-c7c6-497d-b86a-0d2286ccdd73: DOWN {{(pid=91645) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 20 04:33:24.305329 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-be0e0973-ebf8-48c1-b1fc-92cd412d56eb admin admin] Pushing event deleted for resources: {'Port': ['ID=160f1898-b408-446e-8492-c4b331489205,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:24.308392 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-be0e0973-ebf8-48c1-b1fc-92cd412d56eb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:24.402772 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-be0e0973-ebf8-48c1-b1fc-92cd412d56eb admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-8053346'] for router_gateway, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:24.481478 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-895718f3-e797-4b5c-8fd3-2936bb5149c6 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Deleting port ba923399-cb7e-437c-887a-3e86889e7fb9 {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:24.481865 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-895718f3-e797-4b5c-8fd3-2936bb5149c6 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:24.581302 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-be0e0973-ebf8-48c1-b1fc-92cd412d56eb admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-8071641', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-260698'] for router, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:24.583467 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-be0e0973-ebf8-48c1-b1fc-92cd412d56eb admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:24.606751 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-be0e0973-ebf8-48c1-b1fc-92cd412d56eb admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-8056629'] for router, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:24.607530 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-be0e0973-ebf8-48c1-b1fc-92cd412d56eb admin admin] Fanout notify agent at l3_agent the message router_deleted on router 7b630b1e-4360-4f2e-9053-be7e631bfa3a {{(pid=91644) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 20 04:33:24.615629 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-895718f3-e797-4b5c-8fd3-2936bb5149c6 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:24.615955 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-895718f3-e797-4b5c-8fd3-2936bb5149c6 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:24.620059 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-be0e0973-ebf8-48c1-b1fc-92cd412d56eb admin admin] Notify callbacks [] for router, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:24.621162 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-be0e0973-ebf8-48c1-b1fc-92cd412d56eb admin admin] 10.222.0.39 "DELETE /v2.0/routers/7b630b1e-4360-4f2e-9053-be7e631bfa3a HTTP/1.1" status: 204 len: 168 time: 1.3337178 May 20 04:33:24.623765 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43426) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:24.795252 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_v2 [None req-cc37c5ec-6fd8-4338-b9d2-b0f7337c1152 admin admin] Deleting subnet c5423164-3d24-4562-871f-ab03914420dc {{(pid=91644) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 20 04:33:24.798338 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-cc37c5ec-6fd8-4338-b9d2-b0f7337c1152 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:24.833980 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-895718f3-e797-4b5c-8fd3-2936bb5149c6 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Calling delete_port for ba923399-cb7e-437c-887a-3e86889e7fb9 owned by network:floatingip {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:24.850508 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-cc37c5ec-6fd8-4338-b9d2-b0f7337c1152 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-853257'] for subnet, precommit_delete_associations {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:24.934410 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-cc37c5ec-6fd8-4338-b9d2-b0f7337c1152 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-7786687'] for subnet, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:24.946759 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-895718f3-e797-4b5c-8fd3-2936bb5149c6 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:24.951177 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-03ae917e-fc02-4055-aa87-c9b32037327b service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:24.997270 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-895718f3-e797-4b5c-8fd3-2936bb5149c6 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Pushing event deleted for resources: {'Port': ['ID=ba923399-cb7e-437c-887a-3e86889e7fb9,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:25.000529 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-895718f3-e797-4b5c-8fd3-2936bb5149c6 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:25.096399 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-895718f3-e797-4b5c-8fd3-2936bb5149c6 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks [] for floatingip, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:25.117034 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-895718f3-e797-4b5c-8fd3-2936bb5149c6 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:25.119682 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-895718f3-e797-4b5c-8fd3-2936bb5149c6 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] 10.222.0.39 "DELETE /v2.0/floatingips/e600ccfa-c7c6-497d-b86a-0d2286ccdd73 HTTP/1.1" status: 204 len: 168 time: 0.8594272 May 20 04:33:25.206463 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-cc37c5ec-6fd8-4338-b9d2-b0f7337c1152 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-7786632', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-1988925', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-1810229'] for subnet, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:25.211725 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-da79aab3-1e37-4195-9c56-187f531b8bd5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:25.252103 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cc37c5ec-6fd8-4338-b9d2-b0f7337c1152 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=c5423164-3d24-4562-871f-ab03914420dc,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:25.255475 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-cc37c5ec-6fd8-4338-b9d2-b0f7337c1152 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:25.256524 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-cc37c5ec-6fd8-4338-b9d2-b0f7337c1152 admin admin] Notify callbacks [] for subnet, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:25.263848 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-cc37c5ec-6fd8-4338-b9d2-b0f7337c1152 admin admin] 10.222.0.39 "DELETE /v2.0/subnets/c5423164-3d24-4562-871f-ab03914420dc HTTP/1.1" status: 204 len: 168 time: 0.6391230 May 20 04:33:25.266263 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43436) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:25.270125 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-9e96bd0b-fc05-42a6-8716-b500b6eeeca4 None None] Network 9798d104-d4de-413f-b41d-fb9d7e8a818e requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:33:25.374432 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-593607'] for network, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:25.451458 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] Deleting port 617c2db0-1597-44e9-9e82-8ea22dfd47a8 {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:25.451671 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:25.538933 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] DHCP port deletion for 9798d104-d4de-413f-b41d-fb9d7e8a818e request from dv-cinder-782405-9-iscsi {{(pid=91645) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 20 04:33:25.551188 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] For port 617c2db0-1597-44e9-9e82-8ea22dfd47a8, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=617c2db0-1597-44e9-9e82-8ea22dfd47a8,segment=NetworkSegment(d558b057-1c72-4efe-ab53-567b572e2de5),segment_id=d558b057-1c72-4efe-ab53-567b572e2de5)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:33:25.551566 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:25.603380 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] Deleting port 617c2db0-1597-44e9-9e82-8ea22dfd47a8 {{(pid=91645) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:25.603762 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:25.680369 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] Calling delete_port for 617c2db0-1597-44e9-9e82-8ea22dfd47a8 owned by network:dhcp {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:25.703639 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] For port 617c2db0-1597-44e9-9e82-8ea22dfd47a8, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=617c2db0-1597-44e9-9e82-8ea22dfd47a8,segment=NetworkSegment(d558b057-1c72-4efe-ab53-567b572e2de5),segment_id=d558b057-1c72-4efe-ab53-567b572e2de5)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:33:25.704007 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:25.727722 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:25.732283 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-0eb3eb37-5033-47e7-ae2e-fbf8cfc08c22 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:25.774782 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] Pushing event deleted for resources: {'Port': ['ID=617c2db0-1597-44e9-9e82-8ea22dfd47a8,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:25.777912 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:25.778728 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] No DVR routers for this DVR port 617c2db0-1597-44e9-9e82-8ea22dfd47a8 on host dv-cinder-782405-9-iscsi {{(pid=91644) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:33:25.831685 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] Calling delete_port for 617c2db0-1597-44e9-9e82-8ea22dfd47a8 owned by network:dhcp {{(pid=91645) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:25.879134 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716'] for port, after_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:25.882846 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-fa328ca0-d2be-428f-98b5-af1ea68ab950 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:25.925192 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] Pushing event deleted for resources: {'Port': ['ID=617c2db0-1597-44e9-9e82-8ea22dfd47a8,revision_number=None']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:25.929701 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] No DVR routers for this DVR port 617c2db0-1597-44e9-9e82-8ea22dfd47a8 on host dv-cinder-782405-9-iscsi {{(pid=91645) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:33:25.939586 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:25.939586 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:25.939586 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:25.939948 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:25.939948 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:25.940490 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:25.940606 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:25.940952 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:25.941244 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:25.941554 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:25.941886 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:25.942192 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:33:25.945289 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-30759c82-e2aa-4b3b-9f02-c5178e08ab67 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.062s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:25.986973 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-7789108', 'neutron.services.segments.db._delete_segments_for_network-8760688958082'] for network, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:25.999786 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['9798d104-d4de-413f-b41d-fb9d7e8a818e']}} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:26.011660 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'd558b057-1c72-4efe-ab53-567b572e2de5') {'for_net_delete': True} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:26.011956 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'd558b057-1c72-4efe-ab53-567b572e2de5') {} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:26.022998 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8760688968331', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-853278'] for segment, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:26.043531 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:26.051010 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] Releasing vlan 801 on physical network physnet1 to pool {{(pid=91644) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 20 04:33:26.051480 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2001055'] for segment, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: Traceback (most recent call last): May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: timer() May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: cb(*args, **kw) May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: return func(*args, **kwargs) May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: self._notify() May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: self.callback(batched_events) May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: event.method(event) May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: aggregate_id = self._get_aggregate_id(event.segment_id) May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: aggregate_uuid = self.p_client.list_aggregates( May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: return f(self, *a, **k) May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: return self._get(url).json() May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: return self._client.get(url, endpoint_filter=self._ks_filter, May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: return self.request(url, 'GET', **kwargs) May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: base_url = self.get_endpoint(auth, allow=allow, May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: auth = self._auth_required(auth, 'determine endpoint URL') May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 20 04:33:26.065192 dv-cinder-782405-9-iscsi neutron-server[91644]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 20 04:33:26.111966 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device bce609e7-1912-4202-bc3c-9028a36c0adb no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:33:26.138775 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-7789117', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-8053321'] for network, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:26.142948 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:26.146969 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: bce609e7-1912-4202-bc3c-9028a36c0adb {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:33:26.147424 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device bce609e7-1912-4202-bc3c-9028a36c0adb not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:33:26.147828 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 617c2db0-1597-44e9-9e82-8ea22dfd47a8 no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:33:26.156767 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:26.168192 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] Pushing event deleted for resources: {'Network': ['ID=9798d104-d4de-413f-b41d-fb9d7e8a818e,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:26.172190 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.029s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:26.181403 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-e0ca4e0d-f059-438b-a31f-d45c62142ddb admin admin] 10.222.0.39 "DELETE /v2.0/networks/9798d104-d4de-413f-b41d-fb9d7e8a818e HTTP/1.1" status: 204 len: 168 time: 0.9141116 May 20 04:33:26.183456 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: 617c2db0-1597-44e9-9e82-8ea22dfd47a8 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:33:26.183815 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 617c2db0-1597-44e9-9e82-8ea22dfd47a8 not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:33:26.184367 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43440) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:26.184523 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 160f1898-b408-446e-8492-c4b331489205 no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:33:26.218175 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: 160f1898-b408-446e-8492-c4b331489205 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:33:26.218442 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 160f1898-b408-446e-8492-c4b331489205 not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:33:26.724053 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6a0f0699-d603-4e9f-b29a-b6fe43140c15 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-8003145'] for router_interface, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:27.133222 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-6a0f0699-d603-4e9f-b29a-b6fe43140c15 admin admin] Deleting port 8b38f061-73b2-48b5-bd6f-a15baf789252 {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:27.133563 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6a0f0699-d603-4e9f-b29a-b6fe43140c15 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:27.252676 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [None req-6a0f0699-d603-4e9f-b29a-b6fe43140c15 admin admin] For port 8b38f061-73b2-48b5-bd6f-a15baf789252, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=8b38f061-73b2-48b5-bd6f-a15baf789252,segment=NetworkSegment(e3e8d115-0d77-4bf0-a504-52c3fb21936c),segment_id=e3e8d115-0d77-4bf0-a504-52c3fb21936c)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:33:27.253007 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6a0f0699-d603-4e9f-b29a-b6fe43140c15 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:27.462243 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-6a0f0699-d603-4e9f-b29a-b6fe43140c15 admin admin] Calling delete_port for 8b38f061-73b2-48b5-bd6f-a15baf789252 owned by network:router_interface {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:27.629989 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6a0f0699-d603-4e9f-b29a-b6fe43140c15 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:27.634348 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-3cadb8e3-bf38-4112-8219-be0d81ed522e service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:27.677385 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6a0f0699-d603-4e9f-b29a-b6fe43140c15 admin admin] Pushing event deleted for resources: {'Port': ['ID=8b38f061-73b2-48b5-bd6f-a15baf789252,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:27.705692 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-6a0f0699-d603-4e9f-b29a-b6fe43140c15 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.072s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:27.947704 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6a0f0699-d603-4e9f-b29a-b6fe43140c15 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-7924922', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-8054745'] for router_interface, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:28.322247 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-6a0f0699-d603-4e9f-b29a-b6fe43140c15 admin admin] Router 007d36bf-1482-45ff-a392-0fb4d38879a7 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91644) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:33:28.500962 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-6a0f0699-d603-4e9f-b29a-b6fe43140c15 admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:33:28.511876 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-696842bf-74f3-44fd-b4bc-cef6ad730a29 None None] Sync routers for ids ['007d36bf-1482-45ff-a392-0fb4d38879a7'] in dv-cinder-782405-9-iscsi {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:33:28.512377 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-6a0f0699-d603-4e9f-b29a-b6fe43140c15 admin admin] 10.222.0.39 "PUT /v2.0/routers/007d36bf-1482-45ff-a392-0fb4d38879a7/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.3267694 May 20 04:33:28.515165 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43462) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:28.521856 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-696842bf-74f3-44fd-b4bc-cef6ad730a29 None None] Routers {'007d36bf-1482-45ff-a392-0fb4d38879a7'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:33:28.522558 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-696842bf-74f3-44fd-b4bc-cef6ad730a29 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:33:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['007d36bf-1482-45ff-a392-0fb4d38879a7'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:28.692719 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-479708a4-3b98-48d5-9d75-cd50fd49f856 admin admin] Notify callbacks [] for router, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:28.734501 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-696842bf-74f3-44fd-b4bc-cef6ad730a29 None None] Set router 007d36bf-1482-45ff-a392-0fb4d38879a7 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:33:28.974893 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-479708a4-3b98-48d5-9d75-cd50fd49f856 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:29.018156 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-696842bf-74f3-44fd-b4bc-cef6ad730a29 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '007d36bf-1482-45ff-a392-0fb4d38879a7', 'name': 'tempest-TestMinimumBasicScenario-1656849615-router', 'tenant_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.119'}], 'enable_snat': True}, 'gw_port_id': 'a1e201be-75e9-4fbb-b3c4-47eeb5b0221c', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:31Z', 'updated_at': '2021-05-20T04:33:27Z', 'revision_number': 5, 'project_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'gw_port': {'id': 'a1e201be-75e9-4fbb-b3c4-47eeb5b0221c', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6e:9b:ea', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '007d36bf-1482-45ff-a392-0fb4d38879a7', 'device_owner': 'network:router_gateway', 'standard_attr_id': 556, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.119', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:28:31Z', 'updated_at': '2021-05-20T04:28:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:29.021584 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-696842bf-74f3-44fd-b4bc-cef6ad730a29 None None] Checking router: 007d36bf-1482-45ff-a392-0fb4d38879a7 for host: dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:33:29.030110 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-696842bf-74f3-44fd-b4bc-cef6ad730a29 None None] Routers {'007d36bf-1482-45ff-a392-0fb4d38879a7'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:33:29.030712 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-696842bf-74f3-44fd-b4bc-cef6ad730a29 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:33:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['007d36bf-1482-45ff-a392-0fb4d38879a7'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:29.050795 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-479708a4-3b98-48d5-9d75-cd50fd49f856 admin admin] Deleting port a1e201be-75e9-4fbb-b3c4-47eeb5b0221c {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:29.051318 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-479708a4-3b98-48d5-9d75-cd50fd49f856 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:29.160948 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 43474) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:29.181407 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-696842bf-74f3-44fd-b4bc-cef6ad730a29 None None] Set router 007d36bf-1482-45ff-a392-0fb4d38879a7 gateway port host: None {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:33:29.187064 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [None req-479708a4-3b98-48d5-9d75-cd50fd49f856 admin admin] For port a1e201be-75e9-4fbb-b3c4-47eeb5b0221c, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=a1e201be-75e9-4fbb-b3c4-47eeb5b0221c,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:33:29.187561 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-479708a4-3b98-48d5-9d75-cd50fd49f856 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:29.284781 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-696842bf-74f3-44fd-b4bc-cef6ad730a29 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '007d36bf-1482-45ff-a392-0fb4d38879a7', 'name': 'tempest-TestMinimumBasicScenario-1656849615-router', 'tenant_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:31Z', 'updated_at': '2021-05-20T04:33:28Z', 'revision_number': 6, 'project_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'gw_port_host': None}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:29.288777 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-696842bf-74f3-44fd-b4bc-cef6ad730a29 None None] The sync data for ids ['007d36bf-1482-45ff-a392-0fb4d38879a7'] in dv-cinder-782405-9-iscsi is: [{'id': '007d36bf-1482-45ff-a392-0fb4d38879a7', 'name': 'tempest-TestMinimumBasicScenario-1656849615-router', 'tenant_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:28:31Z', 'updated_at': '2021-05-20T04:33:28Z', 'revision_number': 6, 'project_id': '5c8cd90c0a57413e889fa3808d4bdf52', 'gw_port_host': None}] {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:33:29.312314 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-479708a4-3b98-48d5-9d75-cd50fd49f856 admin admin] Calling delete_port for a1e201be-75e9-4fbb-b3c4-47eeb5b0221c owned by network:router_gateway {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:29.454324 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-479708a4-3b98-48d5-9d75-cd50fd49f856 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:29.458836 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-145d1d20-8440-4a55-bf9f-c1334b0362a0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:29.506188 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-479708a4-3b98-48d5-9d75-cd50fd49f856 admin admin] Pushing event deleted for resources: {'Port': ['ID=a1e201be-75e9-4fbb-b3c4-47eeb5b0221c,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:29.531704 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-479708a4-3b98-48d5-9d75-cd50fd49f856 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.073s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:29.597233 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-479708a4-3b98-48d5-9d75-cd50fd49f856 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-8053346'] for router_gateway, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:29.773345 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6ba7175f-d265-42dd-8dec-8031dd21207f admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-8003145'] for router_interface, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:29.774818 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-479708a4-3b98-48d5-9d75-cd50fd49f856 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-8071641', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-260698'] for router, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:29.777075 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-479708a4-3b98-48d5-9d75-cd50fd49f856 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:29.800533 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-479708a4-3b98-48d5-9d75-cd50fd49f856 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-8056629'] for router, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:29.801078 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-479708a4-3b98-48d5-9d75-cd50fd49f856 admin admin] Fanout notify agent at l3_agent the message router_deleted on router 007d36bf-1482-45ff-a392-0fb4d38879a7 {{(pid=91644) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 20 04:33:29.813339 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-479708a4-3b98-48d5-9d75-cd50fd49f856 admin admin] Notify callbacks [] for router, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:29.814423 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-479708a4-3b98-48d5-9d75-cd50fd49f856 admin admin] 10.222.0.39 "DELETE /v2.0/routers/007d36bf-1482-45ff-a392-0fb4d38879a7 HTTP/1.1" status: 204 len: 168 time: 1.2981718 May 20 04:33:29.817082 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43480) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:29.875528 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_v2 [None req-6e2714b0-eb9b-4ab7-9bd8-1fc7eb989f65 admin admin] Deleting subnet ce429852-561a-4890-b1d6-22338214dd01 {{(pid=91644) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 20 04:33:29.878690 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6e2714b0-eb9b-4ab7-9bd8-1fc7eb989f65 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:29.931291 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6e2714b0-eb9b-4ab7-9bd8-1fc7eb989f65 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-853257'] for subnet, precommit_delete_associations {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:30.010102 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6e2714b0-eb9b-4ab7-9bd8-1fc7eb989f65 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-7786687'] for subnet, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:30.146983 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 8b38f061-73b2-48b5-bd6f-a15baf789252 no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:33:30.185988 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: 8b38f061-73b2-48b5-bd6f-a15baf789252 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:33:30.186640 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 8b38f061-73b2-48b5-bd6f-a15baf789252 not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:33:30.187352 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device a1e201be-75e9-4fbb-b3c4-47eeb5b0221c no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:33:30.192569 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-6ba7175f-d265-42dd-8dec-8031dd21207f admin admin] Deleting port 7dc69379-dd61-4add-8e0a-4ef00a211a0a {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:30.192569 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6ba7175f-d265-42dd-8dec-8031dd21207f admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:30.228060 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: a1e201be-75e9-4fbb-b3c4-47eeb5b0221c {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:33:30.228564 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device a1e201be-75e9-4fbb-b3c4-47eeb5b0221c not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:33:30.280222 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6e2714b0-eb9b-4ab7-9bd8-1fc7eb989f65 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-7786632', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-1988925', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-1810229'] for subnet, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:30.286041 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-6c432f4c-92f8-453b-968a-5b23526db9e2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:30.324130 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6e2714b0-eb9b-4ab7-9bd8-1fc7eb989f65 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=ce429852-561a-4890-b1d6-22338214dd01,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:30.327438 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-6e2714b0-eb9b-4ab7-9bd8-1fc7eb989f65 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.041s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:30.328548 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6e2714b0-eb9b-4ab7-9bd8-1fc7eb989f65 admin admin] Notify callbacks [] for subnet, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:30.333688 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e0c4e18c-a5c2-4c65-b19c-45b092d58f58 None None] Network c4d0b06b-6db7-4af5-9ff8-304cb32a79cd requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:33:30.338016 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-6e2714b0-eb9b-4ab7-9bd8-1fc7eb989f65 admin admin] 10.222.0.39 "DELETE /v2.0/subnets/ce429852-561a-4890-b1d6-22338214dd01 HTTP/1.1" status: 204 len: 168 time: 0.5199027 May 20 04:33:30.338781 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43484) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:30.441128 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [None req-6ba7175f-d265-42dd-8dec-8031dd21207f admin admin] For port 7dc69379-dd61-4add-8e0a-4ef00a211a0a, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=7dc69379-dd61-4add-8e0a-4ef00a211a0a,segment=NetworkSegment(b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7),segment_id=b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:33:30.441528 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6ba7175f-d265-42dd-8dec-8031dd21207f admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:30.558026 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-593607'] for network, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:30.567816 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-6ba7175f-d265-42dd-8dec-8031dd21207f admin admin] Calling delete_port for 7dc69379-dd61-4add-8e0a-4ef00a211a0a owned by network:router_interface {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:30.605732 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-eb472546-3579-475a-8577-8f521cc75f1f None None] DHCP port deletion for c4d0b06b-6db7-4af5-9ff8-304cb32a79cd request from dv-cinder-782405-9-iscsi {{(pid=91645) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 20 04:33:30.637354 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] Deleting port 0d0ad457-9322-4fb5-abac-feaec6951e5a {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:30.637690 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:30.673984 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-eb472546-3579-475a-8577-8f521cc75f1f None None] Deleting port 0d0ad457-9322-4fb5-abac-feaec6951e5a {{(pid=91645) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:30.674331 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-eb472546-3579-475a-8577-8f521cc75f1f None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:30.726046 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6ba7175f-d265-42dd-8dec-8031dd21207f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:30.731037 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-1acb4825-0aea-4a81-a5fa-5b0280d88209 tempest-TestShelveInstance-1569987365 tempest-TestShelveInstance-1569987365-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:30.739396 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] For port 0d0ad457-9322-4fb5-abac-feaec6951e5a, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=0d0ad457-9322-4fb5-abac-feaec6951e5a,segment=NetworkSegment(e3e8d115-0d77-4bf0-a504-52c3fb21936c),segment_id=e3e8d115-0d77-4bf0-a504-52c3fb21936c)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:33:30.739772 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:30.773880 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6ba7175f-d265-42dd-8dec-8031dd21207f admin admin] Pushing event deleted for resources: {'Port': ['ID=7dc69379-dd61-4add-8e0a-4ef00a211a0a,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:30.779992 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-6ba7175f-d265-42dd-8dec-8031dd21207f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:30.809189 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-eb472546-3579-475a-8577-8f521cc75f1f None None] For port 0d0ad457-9322-4fb5-abac-feaec6951e5a, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=0d0ad457-9322-4fb5-abac-feaec6951e5a,segment=NetworkSegment(e3e8d115-0d77-4bf0-a504-52c3fb21936c),segment_id=e3e8d115-0d77-4bf0-a504-52c3fb21936c)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:33:30.809573 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-eb472546-3579-475a-8577-8f521cc75f1f None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:30.867638 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] Calling delete_port for 0d0ad457-9322-4fb5-abac-feaec6951e5a owned by network:dhcp {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:30.921169 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:30.931704 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-656f5cc7-c6dd-4fdd-bd5f-044dcb8ac3ea service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:30.937892 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-eb472546-3579-475a-8577-8f521cc75f1f None None] Calling delete_port for 0d0ad457-9322-4fb5-abac-feaec6951e5a owned by network:dhcp {{(pid=91645) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:30.977053 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] No DVR routers for this DVR port 0d0ad457-9322-4fb5-abac-feaec6951e5a on host dv-cinder-782405-9-iscsi {{(pid=91644) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:33:30.989892 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-eb472546-3579-475a-8577-8f521cc75f1f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716'] for port, after_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:30.999001 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c5ba17f1-757a-4b3e-9d67-c2b638d3ec32 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:31.011231 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] Pushing event deleted for resources: {'Port': ['ID=0d0ad457-9322-4fb5-abac-feaec6951e5a,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:31.013796 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.082s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:31.033654 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-eb472546-3579-475a-8577-8f521cc75f1f None None] No DVR routers for this DVR port 0d0ad457-9322-4fb5-abac-feaec6951e5a on host dv-cinder-782405-9-iscsi {{(pid=91645) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:33:31.042499 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-eb472546-3579-475a-8577-8f521cc75f1f None None] Pushing event deleted for resources: {'Port': ['ID=0d0ad457-9322-4fb5-abac-feaec6951e5a,revision_number=None']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:31.044416 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-eb472546-3579-475a-8577-8f521cc75f1f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:31.063250 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6ba7175f-d265-42dd-8dec-8031dd21207f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-7924922', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-8054745'] for router_interface, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:31.103984 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-7789108', 'neutron.services.segments.db._delete_segments_for_network-8760688958082'] for network, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:31.116711 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['c4d0b06b-6db7-4af5-9ff8-304cb32a79cd']}} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:31.133628 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'e3e8d115-0d77-4bf0-a504-52c3fb21936c') {'for_net_delete': True} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:31.133953 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'e3e8d115-0d77-4bf0-a504-52c3fb21936c') {} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:31.145658 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8760688968331', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-853278'] for segment, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:31.161784 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:31.168589 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] Releasing vlan 838 on physical network physnet1 to pool {{(pid=91644) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 20 04:33:31.169055 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2001055'] for segment, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: Traceback (most recent call last): May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: timer() May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: cb(*args, **kw) May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: return func(*args, **kwargs) May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: self._notify() May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: self.callback(batched_events) May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: event.method(event) May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: aggregate_id = self._get_aggregate_id(event.segment_id) May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: aggregate_uuid = self.p_client.list_aggregates( May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: return f(self, *a, **k) May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: return self._get(url).json() May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: return self._client.get(url, endpoint_filter=self._ks_filter, May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: return self.request(url, 'GET', **kwargs) May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: base_url = self.get_endpoint(auth, allow=allow, May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: auth = self._auth_required(auth, 'determine endpoint URL') May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 20 04:33:31.182336 dv-cinder-782405-9-iscsi neutron-server[91644]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 20 04:33:31.256309 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-7789117', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-8053321'] for network, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:31.260033 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:31.273085 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:31.286742 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] Pushing event deleted for resources: {'Network': ['ID=c4d0b06b-6db7-4af5-9ff8-304cb32a79cd,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:31.289765 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.030s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:31.294007 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-38edaf49-72b8-4c3d-9090-a99aa659a123 admin admin] 10.222.0.39 "DELETE /v2.0/networks/c4d0b06b-6db7-4af5-9ff8-304cb32a79cd HTTP/1.1" status: 204 len: 168 time: 0.9513578 May 20 04:33:31.407043 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43490) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:31.425169 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-6ba7175f-d265-42dd-8dec-8031dd21207f admin admin] Router 10aeccef-e762-4811-a411-a1c9817db442 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91643) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:33:31.455200 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-778f53fa-e4f3-45a9-978e-3ea92cbd4a61 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=13d0e12c764f483ea457246b10c4cf7e&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0468938 May 20 04:33:31.457704 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43494) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:31.514925 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-bee2763c-1784-4431-ab3d-1a6d52608ddf admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:31.540376 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-bee2763c-1784-4431-ab3d-1a6d52608ddf admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:31.592903 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-bee2763c-1784-4431-ab3d-1a6d52608ddf admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:31.596601 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-cf5acb11-da9d-42a0-9e39-e53b80b94bfb admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:31.601605 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-6ba7175f-d265-42dd-8dec-8031dd21207f admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:33:31.611471 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bee2763c-1784-4431-ab3d-1a6d52608ddf admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f596c737-91de-4c7f-9aa1-e6e95d668a70,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:31.616789 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-40a197ea-869a-4fe4-b7c1-ead1a64b00b4 None None] Sync routers for ids ['10aeccef-e762-4811-a411-a1c9817db442'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:33:31.617420 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-6ba7175f-d265-42dd-8dec-8031dd21207f admin admin] 10.222.0.39 "PUT /v2.0/routers/10aeccef-e762-4811-a411-a1c9817db442/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.4549010 May 20 04:33:31.617915 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 43498) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:31.620754 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-bee2763c-1784-4431-ab3d-1a6d52608ddf admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.024s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:31.624673 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-bee2763c-1784-4431-ab3d-1a6d52608ddf admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:31.628339 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-bee2763c-1784-4431-ab3d-1a6d52608ddf admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/f596c737-91de-4c7f-9aa1-e6e95d668a70 HTTP/1.1" status: 204 len: 168 time: 0.1672468 May 20 04:33:31.640267 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-40a197ea-869a-4fe4-b7c1-ead1a64b00b4 None None] Routers {'10aeccef-e762-4811-a411-a1c9817db442'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:33:31.641190 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-40a197ea-869a-4fe4-b7c1-ead1a64b00b4 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:33:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['10aeccef-e762-4811-a411-a1c9817db442'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:31.826222 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-40a197ea-869a-4fe4-b7c1-ead1a64b00b4 None None] Set router 10aeccef-e762-4811-a411-a1c9817db442 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:33:31.923392 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c867a753-80c5-4b73-ae2b-0cd17caa9bd7 admin admin] Notify callbacks [] for router, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:32.008164 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43506) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:32.066478 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-4f7634ac-35ba-47df-83ca-165bc43bce9f admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=5c8cd90c0a57413e889fa3808d4bdf52&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0546727 May 20 04:33:32.067098 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43510) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:32.141200 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e1a6a008-61fd-46da-a558-d1db426d91e7 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:32.152715 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c867a753-80c5-4b73-ae2b-0cd17caa9bd7 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:32.154390 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-40a197ea-869a-4fe4-b7c1-ead1a64b00b4 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33'}], 'enable_snat': True}, 'gw_port_id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:33:30Z', 'revision_number': 5, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port': {'id': 'c3a0d63d-eb15-4b80-a8a4-15678c28b534', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:64:1f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '10aeccef-e762-4811-a411-a1c9817db442', 'device_owner': 'network:router_gateway', 'standard_attr_id': 295, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.33', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:31Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:32.158353 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-40a197ea-869a-4fe4-b7c1-ead1a64b00b4 None None] Checking router: 10aeccef-e762-4811-a411-a1c9817db442 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:33:32.169026 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-40a197ea-869a-4fe4-b7c1-ead1a64b00b4 None None] Routers {'10aeccef-e762-4811-a411-a1c9817db442'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:33:32.169703 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-40a197ea-869a-4fe4-b7c1-ead1a64b00b4 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:33:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['10aeccef-e762-4811-a411-a1c9817db442'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:32.171842 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e1a6a008-61fd-46da-a558-d1db426d91e7 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:32.235258 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 0d0ad457-9322-4fb5-abac-feaec6951e5a no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:33:32.237062 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e1a6a008-61fd-46da-a558-d1db426d91e7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:32.242586 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-62c2fe2a-24ac-43c3-8a93-9db183581047 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:32.251167 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-c867a753-80c5-4b73-ae2b-0cd17caa9bd7 admin admin] Deleting port c3a0d63d-eb15-4b80-a8a4-15678c28b534 {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:32.251526 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c867a753-80c5-4b73-ae2b-0cd17caa9bd7 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:32.252976 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e1a6a008-61fd-46da-a558-d1db426d91e7 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=4fcfb45d-4ed7-4de4-a623-ae5bf3006001,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:32.256474 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e1a6a008-61fd-46da-a558-d1db426d91e7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.014s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:32.260424 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e1a6a008-61fd-46da-a558-d1db426d91e7 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:32.263266 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-e1a6a008-61fd-46da-a558-d1db426d91e7 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/4fcfb45d-4ed7-4de4-a623-ae5bf3006001 HTTP/1.1" status: 204 len: 168 time: 0.1938512 May 20 04:33:32.341426 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: 0d0ad457-9322-4fb5-abac-feaec6951e5a {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:33:32.341829 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 0d0ad457-9322-4fb5-abac-feaec6951e5a not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:33:32.377562 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-40a197ea-869a-4fe4-b7c1-ead1a64b00b4 None None] Set router 10aeccef-e762-4811-a411-a1c9817db442 gateway port host: None {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:33:32.379125 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [None req-c867a753-80c5-4b73-ae2b-0cd17caa9bd7 admin admin] For port c3a0d63d-eb15-4b80-a8a4-15678c28b534, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=c3a0d63d-eb15-4b80-a8a4-15678c28b534,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:33:32.379825 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c867a753-80c5-4b73-ae2b-0cd17caa9bd7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:32.490002 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-40a197ea-869a-4fe4-b7c1-ead1a64b00b4 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:33:32Z', 'revision_number': 6, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port_host': None}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:32.494571 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-40a197ea-869a-4fe4-b7c1-ead1a64b00b4 None None] The sync data for ids ['10aeccef-e762-4811-a411-a1c9817db442'] in dv-cinder-782405-9-iscsi is: [{'id': '10aeccef-e762-4811-a411-a1c9817db442', 'name': 'tempest-AttachVolumeTestJSON-97673996-router', 'tenant_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:30Z', 'updated_at': '2021-05-20T04:33:32Z', 'revision_number': 6, 'project_id': '198bc8ab083f4a729f32b8ef27a6ee77', 'gw_port_host': None}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:33:32.509226 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-c867a753-80c5-4b73-ae2b-0cd17caa9bd7 admin admin] Calling delete_port for c3a0d63d-eb15-4b80-a8a4-15678c28b534 owned by network:router_gateway {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:32.662815 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c867a753-80c5-4b73-ae2b-0cd17caa9bd7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:32.667328 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-895718f3-e797-4b5c-8fd3-2936bb5149c6 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:32.709256 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c867a753-80c5-4b73-ae2b-0cd17caa9bd7 admin admin] Pushing event deleted for resources: {'Port': ['ID=c3a0d63d-eb15-4b80-a8a4-15678c28b534,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:32.712307 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-c867a753-80c5-4b73-ae2b-0cd17caa9bd7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:32.801108 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c867a753-80c5-4b73-ae2b-0cd17caa9bd7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-8053346'] for router_gateway, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:32.982917 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c867a753-80c5-4b73-ae2b-0cd17caa9bd7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-8071641', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-260698'] for router, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:32.985187 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c867a753-80c5-4b73-ae2b-0cd17caa9bd7 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:33.072834 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c867a753-80c5-4b73-ae2b-0cd17caa9bd7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-8056629'] for router, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:33.073450 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-c867a753-80c5-4b73-ae2b-0cd17caa9bd7 admin admin] Fanout notify agent at l3_agent the message router_deleted on router 10aeccef-e762-4811-a411-a1c9817db442 {{(pid=91643) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 20 04:33:33.088039 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c867a753-80c5-4b73-ae2b-0cd17caa9bd7 admin admin] Notify callbacks [] for router, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:33.089813 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-c867a753-80c5-4b73-ae2b-0cd17caa9bd7 admin admin] 10.222.0.39 "DELETE /v2.0/routers/10aeccef-e762-4811-a411-a1c9817db442 HTTP/1.1" status: 204 len: 168 time: 1.4704206 May 20 04:33:33.091756 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 43516) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:33.150886 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_v2 [None req-2720e24f-e6ee-4831-9dc1-edc5f7bc2800 admin admin] Deleting subnet 4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3 {{(pid=91643) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 20 04:33:33.153847 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2720e24f-e6ee-4831-9dc1-edc5f7bc2800 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:33.203001 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2720e24f-e6ee-4831-9dc1-edc5f7bc2800 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-1472754'] for subnet, precommit_delete_associations {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:33.319905 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2720e24f-e6ee-4831-9dc1-edc5f7bc2800 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-7786687'] for subnet, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:33.678231 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2720e24f-e6ee-4831-9dc1-edc5f7bc2800 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-7786632', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2050499', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2172490'] for subnet, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:33.682271 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:33.715589 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2720e24f-e6ee-4831-9dc1-edc5f7bc2800 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:33.718825 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-2720e24f-e6ee-4831-9dc1-edc5f7bc2800 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.037s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:33.719789 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2720e24f-e6ee-4831-9dc1-edc5f7bc2800 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:33.726051 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-12582b9c-3167-4e24-af43-0132e3798e96 None None] Network 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:33:33.726966 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-2720e24f-e6ee-4831-9dc1-edc5f7bc2800 admin admin] 10.222.0.39 "DELETE /v2.0/subnets/4727b8f1-54ac-4d4d-8ea3-d2b5aa3690a3 HTTP/1.1" status: 204 len: 168 time: 0.6331143 May 20 04:33:33.730775 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43520) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:33.836443 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-593607'] for network, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:33.909654 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] Deleting port 4f7409b5-c762-4162-ae2d-d2066230c4f4 {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:33.909960 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:34.009738 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] For port 4f7409b5-c762-4162-ae2d-d2066230c4f4, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=4f7409b5-c762-4162-ae2d-d2066230c4f4,segment=NetworkSegment(b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7),segment_id=b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:33:34.010090 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:34.015094 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-a7be9ad6-f638-46e4-960b-0da20a525f20 None None] DHCP port deletion for 49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d request from dv-cinder-782405-9-iscsi {{(pid=91645) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 20 04:33:34.085778 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-a7be9ad6-f638-46e4-960b-0da20a525f20 None None] Deleting port 4f7409b5-c762-4162-ae2d-d2066230c4f4 {{(pid=91645) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:34.086176 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-a7be9ad6-f638-46e4-960b-0da20a525f20 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:34.121138 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 7dc69379-dd61-4add-8e0a-4ef00a211a0a no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:33:34.133518 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] Calling delete_port for 4f7409b5-c762-4162-ae2d-d2066230c4f4 owned by network:dhcp {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:34.183588 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:34.188524 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-07d1522d-d8f5-4616-a208-44d870d341bc tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:34.205004 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: 7dc69379-dd61-4add-8e0a-4ef00a211a0a {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:33:34.205352 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 7dc69379-dd61-4add-8e0a-4ef00a211a0a not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:33:34.205776 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 4f7409b5-c762-4162-ae2d-d2066230c4f4 no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:33:34.231837 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] Pushing event deleted for resources: {'Port': ['ID=4f7409b5-c762-4162-ae2d-d2066230c4f4,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:34.233482 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] No DVR routers for this DVR port 4f7409b5-c762-4162-ae2d-d2066230c4f4 on host dv-cinder-782405-9-iscsi {{(pid=91644) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:33:34.234905 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:34.248859 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: 4f7409b5-c762-4162-ae2d-d2066230c4f4 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:33:34.249207 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 4f7409b5-c762-4162-ae2d-d2066230c4f4 not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:33:34.249627 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device c3a0d63d-eb15-4b80-a8a4-15678c28b534 no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:33:34.280402 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-a7be9ad6-f638-46e4-960b-0da20a525f20 None None] For port 4f7409b5-c762-4162-ae2d-d2066230c4f4, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=4f7409b5-c762-4162-ae2d-d2066230c4f4,segment=NetworkSegment(b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7),segment_id=b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:33:34.280798 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-a7be9ad6-f638-46e4-960b-0da20a525f20 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:34.297248 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: c3a0d63d-eb15-4b80-a8a4-15678c28b534 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:33:34.297575 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device c3a0d63d-eb15-4b80-a8a4-15678c28b534 not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:33:34.346084 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-7789108', 'neutron.services.segments.db._delete_segments_for_network-8760688958082'] for network, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:34.359408 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d']}} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:34.372012 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7') {'for_net_delete': True} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:34.372309 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'b0a6cb40-23cc-4fb9-9d8e-09a6f8dc1fb7') {} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:34.384060 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8760688968331', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-853278'] for segment, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:34.400586 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:34.407158 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] Releasing vlan 808 on physical network physnet1 to pool {{(pid=91644) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 20 04:33:34.407667 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2001055'] for segment, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:34.407871 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-a7be9ad6-f638-46e4-960b-0da20a525f20 None None] Calling delete_port for 4f7409b5-c762-4162-ae2d-d2066230c4f4 owned by network:dhcp {{(pid=91645) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: Traceback (most recent call last): May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: timer() May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: cb(*args, **kw) May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: return func(*args, **kwargs) May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: self._notify() May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: self.callback(batched_events) May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: event.method(event) May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: aggregate_id = self._get_aggregate_id(event.segment_id) May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: aggregate_uuid = self.p_client.list_aggregates( May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: return f(self, *a, **k) May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: return self._get(url).json() May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: return self._client.get(url, endpoint_filter=self._ks_filter, May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: return self.request(url, 'GET', **kwargs) May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: base_url = self.get_endpoint(auth, allow=allow, May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: auth = self._auth_required(auth, 'determine endpoint URL') May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 20 04:33:34.420597 dv-cinder-782405-9-iscsi neutron-server[91644]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 20 04:33:34.451350 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-a7be9ad6-f638-46e4-960b-0da20a525f20 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716'] for port, after_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:34.454955 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:34.498739 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a7be9ad6-f638-46e4-960b-0da20a525f20 None None] Pushing event deleted for resources: {'Port': ['ID=4f7409b5-c762-4162-ae2d-d2066230c4f4,revision_number=None']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:34.501074 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-a7be9ad6-f638-46e4-960b-0da20a525f20 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:34.503637 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-a7be9ad6-f638-46e4-960b-0da20a525f20 None None] No DVR routers for this DVR port 4f7409b5-c762-4162-ae2d-d2066230c4f4 on host dv-cinder-782405-9-iscsi {{(pid=91645) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:33:34.590932 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-7789117', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-8053321'] for network, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:34.594928 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:34.607852 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:34.617904 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] Pushing event deleted for resources: {'Network': ['ID=49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:34.620975 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:34.633670 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-f4ef9128-a7d9-4832-911f-46b1a343f9e1 admin admin] 10.222.0.39 "DELETE /v2.0/networks/49a0be7b-c49e-4c8d-b4a7-9bc9a86a3d0d HTTP/1.1" status: 204 len: 168 time: 0.9017944 May 20 04:33:34.822133 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43526) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:34.870447 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-014cff4f-0cf8-4fbc-9bb1-d6eff689dd31 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=198bc8ab083f4a729f32b8ef27a6ee77&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0470216 May 20 04:33:34.874035 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 43530) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:34.945860 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bc764e6a-63f5-4a8f-b0a6-c4698968dab7 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:34.971700 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bc764e6a-63f5-4a8f-b0a6-c4698968dab7 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:35.027486 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bc764e6a-63f5-4a8f-b0a6-c4698968dab7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:35.032315 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-19e16154-7f87-49b5-98b5-55a35496478e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:35.046735 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bc764e6a-63f5-4a8f-b0a6-c4698968dab7 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f17ceb35-5276-4876-9705-17e2c336a41c,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:35.049627 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-bc764e6a-63f5-4a8f-b0a6-c4698968dab7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:35.051763 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bc764e6a-63f5-4a8f-b0a6-c4698968dab7 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:35.053153 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-bc764e6a-63f5-4a8f-b0a6-c4698968dab7 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/f17ceb35-5276-4876-9705-17e2c336a41c HTTP/1.1" status: 204 len: 168 time: 0.1778529 May 20 04:33:35.662998 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:38.204507 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fa768455-5066-4366-aca1-dd848b99e524 req-f82ac9b0-055c-4098-bf39-0f34ca867350 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:38.205940 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fa768455-5066-4366-aca1-dd848b99e524 req-f82ac9b0-055c-4098-bf39-0f34ca867350 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 946 time: 0.0817645 May 20 04:33:38.260260 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fa768455-5066-4366-aca1-dd848b99e524 req-ed54bb06-b159-4d59-92af-560bf8fa96a3 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0483680 May 20 04:33:39.481204 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b1e74eb-efcc-4679-a483-766fa591a945 req-45cd303a-1a0f-4a50-b1b3-58ffbdee1b44 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:39.482555 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0b1e74eb-efcc-4679-a483-766fa591a945 req-45cd303a-1a0f-4a50-b1b3-58ffbdee1b44 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 946 time: 0.0969141 May 20 04:33:39.537445 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-0b1e74eb-efcc-4679-a483-766fa591a945 req-3bb3c98e-70cf-4c94-aec4-4d5c803848e1 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0460515 May 20 04:33:40.746752 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1676b0de-0c11-46bc-a93d-c6c7393fff0a req-38a37c31-9115-4d68-a099-bb97e4848f62 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:40.748187 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1676b0de-0c11-46bc-a93d-c6c7393fff0a req-38a37c31-9115-4d68-a099-bb97e4848f62 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 946 time: 0.0886328 May 20 04:33:40.796631 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1676b0de-0c11-46bc-a93d-c6c7393fff0a req-73ea24c3-c71d-43b5-ae4b-c28c8246bb4c tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0396142 May 20 04:33:42.027051 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1bdbccc0-be8c-4638-9f11-ee205606b3f3 req-ed772dd9-2755-4e43-aeee-7d9590e830db tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:42.028565 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1bdbccc0-be8c-4638-9f11-ee205606b3f3 req-ed772dd9-2755-4e43-aeee-7d9590e830db tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 946 time: 0.0943563 May 20 04:33:42.075357 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1bdbccc0-be8c-4638-9f11-ee205606b3f3 req-69d7c246-374f-429f-82e1-ffc74e5b8012 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0409188 May 20 04:33:42.627319 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-535d91a4-40a4-4414-83d7-2320af8100e4 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:43.276927 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5b9b1d16-70ee-4609-bd3f-fcc04428fcd1 req-58f7db99-721c-47b5-9d9d-2f40f96bb2a5 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:43.278640 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5b9b1d16-70ee-4609-bd3f-fcc04428fcd1 req-58f7db99-721c-47b5-9d9d-2f40f96bb2a5 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 946 time: 0.0924056 May 20 04:33:43.325114 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-5b9b1d16-70ee-4609-bd3f-fcc04428fcd1 req-2cce9561-68dc-4a3c-ada9-9842ca071d9d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups?id=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0404785 May 20 04:33:43.734920 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1def9535-72d2-46da-a70b-da2ef430725e req-7aaa677f-26c4-4735-9f28-45ea67e33e00 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:43.736274 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1def9535-72d2-46da-a70b-da2ef430725e req-7aaa677f-26c4-4735-9f28-45ea67e33e00 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.74 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 946 time: 0.0880628 May 20 04:33:43.870177 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-1def9535-72d2-46da-a70b-da2ef430725e req-ac9486cb-1d11-414b-a7ed-422ff6bab157 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Deleting port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:43.870532 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-1def9535-72d2-46da-a70b-da2ef430725e req-ac9486cb-1d11-414b-a7ed-422ff6bab157 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:44.001922 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-1def9535-72d2-46da-a70b-da2ef430725e req-ac9486cb-1d11-414b-a7ed-422ff6bab157 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] For port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:33:44.002157 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-1def9535-72d2-46da-a70b-da2ef430725e req-ac9486cb-1d11-414b-a7ed-422ff6bab157 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:44.293278 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-1def9535-72d2-46da-a70b-da2ef430725e req-ac9486cb-1d11-414b-a7ed-422ff6bab157 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for floatingip, precommit_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:44.294050 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-1def9535-72d2-46da-a70b-da2ef430725e req-ac9486cb-1d11-414b-a7ed-422ff6bab157 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-8003251'] for floatingip, after_update {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:44.294050 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.l3_db [req-1def9535-72d2-46da-a70b-da2ef430725e req-ac9486cb-1d11-414b-a7ed-422ff6bab157 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Floating IP 7134e5d9-58ed-4bdf-88ad-c3f9e36c5a1c disassociated. External IP: 172.27.0.70, port: b1d353b0-0c40-42fa-a24f-0e337c3c1d7c. May 20 04:33:44.294257 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-1def9535-72d2-46da-a70b-da2ef430725e req-ac9486cb-1d11-414b-a7ed-422ff6bab157 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Calling delete_port for b1d353b0-0c40-42fa-a24f-0e337c3c1d7c owned by compute:nova {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:44.404760 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-1def9535-72d2-46da-a70b-da2ef430725e req-ac9486cb-1d11-414b-a7ed-422ff6bab157 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:44.410564 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-adade246-1699-46dc-8be2-7be83e659574 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:44.571525 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ac9486cb-1d11-414b-a7ed-422ff6bab157 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Pushing event deleted for resources: {'Port': ['ID=b1d353b0-0c40-42fa-a24f-0e337c3c1d7c,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:44.579981 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-ac9486cb-1d11-414b-a7ed-422ff6bab157 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.169s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:44.586865 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-42f39069-2bcd-47a2-acdb-2d1460525323 req-f9633c2b-2e54-4736-9fc3-4c4d3b6ef46f tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/ports?device_id=74d085a4-f817-48d6-b8d8-8504cc4ca99d HTTP/1.1" status: 200 len: 210 time: 0.1388350 May 20 04:33:44.776581 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [req-1def9535-72d2-46da-a70b-da2ef430725e req-ac9486cb-1d11-414b-a7ed-422ff6bab157 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91643) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:33:44.965028 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-1def9535-72d2-46da-a70b-da2ef430725e req-ac9486cb-1d11-414b-a7ed-422ff6bab157 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:33:44.981762 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b2618d7a-b500-4477-855b-beb81e612517 None None] Sync routers for ids ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'] in dv-cinder-782405-9-iscsi {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:33:44.990977 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:44.996798 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b2618d7a-b500-4477-855b-beb81e612517 None None] Routers {'5a3f69b9-fd58-499f-bec9-75215cd7f8fc'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:33:44.997435 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-b2618d7a-b500-4477-855b-beb81e612517 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 4, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:33:15Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:45.041199 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [req-1def9535-72d2-46da-a70b-da2ef430725e req-ac9486cb-1d11-414b-a7ed-422ff6bab157 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] No DVR routers for this DVR port b1d353b0-0c40-42fa-a24f-0e337c3c1d7c on host hv-cinder-78240 {{(pid=91643) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:33:45.043924 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-1def9535-72d2-46da-a70b-da2ef430725e req-ac9486cb-1d11-414b-a7ed-422ff6bab157 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:45.046712 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '74d085a4-f817-48d6-b8d8-8504cc4ca99d', 'name': 'network-vif-deleted', 'tag': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:33:45.059720 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1def9535-72d2-46da-a70b-da2ef430725e req-ac9486cb-1d11-414b-a7ed-422ff6bab157 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.74 "DELETE /v2.0/ports/b1d353b0-0c40-42fa-a24f-0e337c3c1d7c HTTP/1.1" status: 204 len: 173 time: 1.3174291 May 20 04:33:45.116050 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-91f21960-9f20-4d4d-adb7-c0538bf687a8 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:45.122304 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '74d085a4-f817-48d6-b8d8-8504cc4ca99d', 'name': 'network-vif-deleted', 'tag': 'b1d353b0-0c40-42fa-a24f-0e337c3c1d7c', 'status': 'completed', 'code': 200} May 20 04:33:45.194365 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-b2618d7a-b500-4477-855b-beb81e612517 None None] Set router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:33:45.504330 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-b2618d7a-b500-4477-855b-beb81e612517 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port': {'id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:e4:9a:7c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:39Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'ac9495b7-5868-4104-85c9-3567f1350467', 'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'mac_address': 'fa:16:3e:12:5b:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_interface', 'standard_attr_id': 82, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:45Z', 'updated_at': '2021-05-20T04:21:55Z', 'revision_number': 5, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'subnets': [{'id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:45.508376 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b2618d7a-b500-4477-855b-beb81e612517 None None] Checking router: 5a3f69b9-fd58-499f-bec9-75215cd7f8fc for host: dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:33:45.516786 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b2618d7a-b500-4477-855b-beb81e612517 None None] Routers {'5a3f69b9-fd58-499f-bec9-75215cd7f8fc'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:33:45.517330 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-b2618d7a-b500-4477-855b-beb81e612517 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:33:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:45.716831 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-b2618d7a-b500-4477-855b-beb81e612517 None None] Set router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:33:45.723352 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 43656) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:45.787756 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a48ef099-fc5b-4820-83ee-7349dfd6c098 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:45.820503 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a48ef099-fc5b-4820-83ee-7349dfd6c098 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:45.877545 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a48ef099-fc5b-4820-83ee-7349dfd6c098 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:45.883023 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-40e862ce-9dd8-4cbe-a4a9-f10da47d5364 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:45.898786 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a48ef099-fc5b-4820-83ee-7349dfd6c098 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=7a1d2a6b-5b70-4cb9-a737-fa50141ab0be,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:45.901371 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-a48ef099-fc5b-4820-83ee-7349dfd6c098 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:45.913947 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a48ef099-fc5b-4820-83ee-7349dfd6c098 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:45.918121 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43666) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:45.918611 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-a48ef099-fc5b-4820-83ee-7349dfd6c098 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "DELETE /v2.0/security-groups/7a1d2a6b-5b70-4cb9-a737-fa50141ab0be HTTP/1.1" status: 204 len: 168 time: 0.1938150 May 20 04:33:45.951814 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.api.v2.resource [None req-03616060-8f5a-4592-a3f9-57803346d79e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] show failed (client error): The resource could not be found. May 20 04:33:45.952911 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-03616060-8f5a-4592-a3f9-57803346d79e tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "GET /v2.0/security-groups/7a1d2a6b-5b70-4cb9-a737-fa50141ab0be HTTP/1.1" status: 404 len: 347 time: 0.0330768 May 20 04:33:45.955818 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43672) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:46.020706 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:33:46.037176 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-b2618d7a-b500-4477-855b-beb81e612517 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port': {'id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:e4:9a:7c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:39Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'ac9495b7-5868-4104-85c9-3567f1350467', 'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'mac_address': 'fa:16:3e:12:5b:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_interface', 'standard_attr_id': 82, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:45Z', 'updated_at': '2021-05-20T04:21:55Z', 'revision_number': 5, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'subnets': [{'id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:46.041231 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b2618d7a-b500-4477-855b-beb81e612517 None None] The sync data for ids ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'] in dv-cinder-782405-9-iscsi is: [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port': {'id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:e4:9a:7c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:39Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': 'ac9495b7-5868-4104-85c9-3567f1350467', 'name': '', 'network_id': '64bcea8a-6d35-406b-a823-fa1185008174', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'mac_address': 'fa:16:3e:12:5b:da', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_interface', 'standard_attr_id': 82, 'fixed_ips': [{'subnet_id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:45Z', 'updated_at': '2021-05-20T04:21:55Z', 'revision_number': 5, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'subnets': [{'id': '203a3962-2282-4f51-ac92-c9695b5c0cfe', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:33:46.189001 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-a76b07f4-2bf6-4b9b-9324-952406b5fe1d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Deleting port c8308b42-b870-4612-be13-022cf48bcb41 {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:46.189368 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a76b07f4-2bf6-4b9b-9324-952406b5fe1d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:46.232441 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:46.279968 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a5cca9e5-4dc2-42a2-8227-d45dc19e9c25 None None] New status for floating IP 7134e5d9-58ed-4bdf-88ad-c3f9e36c5a1c: DOWN {{(pid=91645) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 20 04:33:46.319723 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a76b07f4-2bf6-4b9b-9324-952406b5fe1d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:46.320298 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a76b07f4-2bf6-4b9b-9324-952406b5fe1d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:46.377646 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 43684) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:46.450768 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-a76b07f4-2bf6-4b9b-9324-952406b5fe1d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Calling delete_port for c8308b42-b870-4612-be13-022cf48bcb41 owned by network:floatingip {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:46.500496 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8f46111c-7a46-4921-894c-48d10490c6a1 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:46.516306 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-8f46111c-7a46-4921-894c-48d10490c6a1 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:33:46.520178 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-8f46111c-7a46-4921-894c-48d10490c6a1 admin admin] Usage tracker for resource:security_group_rule and tenant:b05b1ce1c9dc4ae99ecf993fb7bda352 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:33:46.522938 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-8f46111c-7a46-4921-894c-48d10490c6a1 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:33:46.528898 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-8f46111c-7a46-4921-894c-48d10490c6a1 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:33:46.565814 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a76b07f4-2bf6-4b9b-9324-952406b5fe1d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:46.570392 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-aebf0826-3b92-4fc4-9bd5-789bcff7aa00 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:46.615943 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a76b07f4-2bf6-4b9b-9324-952406b5fe1d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Pushing event deleted for resources: {'Port': ['ID=c8308b42-b870-4612-be13-022cf48bcb41,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:46.639945 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8f46111c-7a46-4921-894c-48d10490c6a1 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:46.644405 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-a76b07f4-2bf6-4b9b-9324-952406b5fe1d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:46.644865 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-8f46111c-7a46-4921-894c-48d10490c6a1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:46.653382 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-be19a9d7-e273-4a1e-84cf-984ac9d58fc3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:46.695763 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-8f46111c-7a46-4921-894c-48d10490c6a1 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=b05b1ce1c9dc4ae99ecf993fb7bda352&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3141994 May 20 04:33:46.695763 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8f46111c-7a46-4921-894c-48d10490c6a1 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=424028e0-deea-41d4-a794-b167849e09c6,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:46.696423 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 43690) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:46.698709 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-8f46111c-7a46-4921-894c-48d10490c6a1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:46.707769 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a76b07f4-2bf6-4b9b-9324-952406b5fe1d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks [] for floatingip, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:46.721858 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a76b07f4-2bf6-4b9b-9324-952406b5fe1d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:46.723404 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-a76b07f4-2bf6-4b9b-9324-952406b5fe1d tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] 10.222.0.39 "DELETE /v2.0/floatingips/7134e5d9-58ed-4bdf-88ad-c3f9e36c5a1c HTTP/1.1" status: 204 len: 168 time: 0.7660935 May 20 04:33:46.759980 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2f8c5c25-95b9-401b-8c1f-867c018e9785 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:46.785423 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2f8c5c25-95b9-401b-8c1f-867c018e9785 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:46.835872 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2f8c5c25-95b9-401b-8c1f-867c018e9785 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:46.840657 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-67526a2a-c7da-4fb8-8e4b-feca7051d709 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:46.852706 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2f8c5c25-95b9-401b-8c1f-867c018e9785 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=424028e0-deea-41d4-a794-b167849e09c6,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:46.855639 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-2f8c5c25-95b9-401b-8c1f-867c018e9785 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:46.857611 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2f8c5c25-95b9-401b-8c1f-867c018e9785 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:46.858680 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-2f8c5c25-95b9-401b-8c1f-867c018e9785 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/424028e0-deea-41d4-a794-b167849e09c6 HTTP/1.1" status: 204 len: 168 time: 0.1614134 May 20 04:33:47.267127 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b1d353b0-0c40-42fa-a24f-0e337c3c1d7c no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:33:47.301198 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: b1d353b0-0c40-42fa-a24f-0e337c3c1d7c {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:33:47.301549 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b1d353b0-0c40-42fa-a24f-0e337c3c1d7c not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:33:47.578120 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 43718) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:47.690183 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Request body: {'network': {'name': 'tempest-VolumesActionsTest-1324382408-network', 'tenant_id': 'bc0ec8722e724a239e6514c8a40b6684'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:33:47.695250 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:33:47.699477 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Usage tracker for resource:network and tenant:bc0ec8722e724a239e6514c8a40b6684 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:33:47.702502 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:33:47.708671 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:33:47.722975 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Made reservation on behalf of bc0ec8722e724a239e6514c8a40b6684 for: {'network': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:33:47.725260 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-585852'] for network, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:47.737612 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:47.752298 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:33:47.756557 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Usage tracker for resource:security_group_rule and tenant:bc0ec8722e724a239e6514c8a40b6684 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:33:47.759572 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:33:47.765791 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:33:47.851197 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:47.855927 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:47.858810 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-2f8c5c25-95b9-401b-8c1f-867c018e9785 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:47.867464 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:47.891826 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 845} {{(pid=91643) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 20 04:33:47.900317 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=24943297-64ff-4c69-b9ca-ea881f47715e,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:47.903513 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:47.907601 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:47.908477 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.segments_db [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Added segment 7ae61a20-156c-49d5-97f4-da7391c99834 of type vlan for network 4155c7fb-741f-47f2-a8b6-c4b7287d1841 May 20 04:33:47.933762 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8760688516720'] for network, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:47.964012 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915'] for network, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:47.965341 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:47.965913 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:47.969560 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-985a87e6-5d0c-4aa8-947d-407ef0f66109 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:48.067668 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Pushing event updated for resources: {'Network': ['ID=4155c7fb-741f-47f2-a8b6-c4b7287d1841,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:48.074681 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] 10.222.0.39 "POST /v2.0/networks HTTP/1.1" status: 201 len: 882 time: 0.4919567 May 20 04:33:48.074681 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-857b075d-5908-4991-a8f7-f4c0e2e296e6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.102s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:48.074681 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 43728) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:48.081163 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-820282ee-fea7-4ea5-a22c-05bcbd327dcc admin admin] Request body: {'subnet': {'network_id': '4155c7fb-741f-47f2-a8b6-c4b7287d1841', 'cidr': '172.20.0.0/28', 'name': 'tempest-VolumesActionsTest-1324382408-subnet', 'tenant_id': 'bc0ec8722e724a239e6514c8a40b6684', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:33:48.086137 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Network 4155c7fb-741f-47f2-a8b6-c4b7287d1841 details requested by agent ovs-agent-dv-cinder-782405-9-iscsi with host dv-cinder-782405-9-iscsi {{(pid=91645) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 20 04:33:48.087572 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-820282ee-fea7-4ea5-a22c-05bcbd327dcc admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:33:48.091521 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-820282ee-fea7-4ea5-a22c-05bcbd327dcc admin admin] Usage tracker for resource:subnet and tenant:bc0ec8722e724a239e6514c8a40b6684 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:33:48.094682 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-820282ee-fea7-4ea5-a22c-05bcbd327dcc admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:33:48.100418 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-820282ee-fea7-4ea5-a22c-05bcbd327dcc admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:33:48.114589 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-820282ee-fea7-4ea5-a22c-05bcbd327dcc admin admin] Made reservation on behalf of bc0ec8722e724a239e6514c8a40b6684 for: {'subnet': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:33:48.117153 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-820282ee-fea7-4ea5-a22c-05bcbd327dcc admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2172529'] for subnet, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:48.506695 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-820282ee-fea7-4ea5-a22c-05bcbd327dcc admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-2050444', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2172483'] for subnet, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:48.510337 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-83eea8cc-0fe2-4097-b6ac-04d4078f9c56 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:48.684840 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-820282ee-fea7-4ea5-a22c-05bcbd327dcc admin admin] Pushing event updated for resources: {'Subnet': ['ID=e23e84bb-81f2-40ad-8891-cdc124efa6cd,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:48.689174 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-820282ee-fea7-4ea5-a22c-05bcbd327dcc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.179s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:48.694191 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-820282ee-fea7-4ea5-a22c-05bcbd327dcc admin admin] Checking segment: {'id': '7ae61a20-156c-49d5-97f4-da7391c99834', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 845, 'network_id': '4155c7fb-741f-47f2-a8b6-c4b7287d1841'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91643) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:33:48.694573 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-820282ee-fea7-4ea5-a22c-05bcbd327dcc admin admin] Network hostable DHCP agents. Network: 4155c7fb-741f-47f2-a8b6-c4b7287d1841, hostable agents: ['fffae25a-f450-4182-8f9e-ac29d143f6c9'], hosted agents: [] {{(pid=91643) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 20 04:33:48.712821 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-820282ee-fea7-4ea5-a22c-05bcbd327dcc admin admin] Network 4155c7fb-741f-47f2-a8b6-c4b7287d1841 is scheduled to be hosted by DHCP agent fffae25a-f450-4182-8f9e-ac29d143f6c9 with binding_index 1 {{(pid=91643) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 20 04:33:48.722384 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-820282ee-fea7-4ea5-a22c-05bcbd327dcc admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-782405-9-iscsi, 4155c7fb-741f-47f2-a8b6-c4b7287d1841) {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:33:48.730834 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-820282ee-fea7-4ea5-a22c-05bcbd327dcc admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:48.730995 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-42186d56-4c9f-439c-986a-b27688951bb6 None None] Network 4155c7fb-741f-47f2-a8b6-c4b7287d1841 requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:33:48.731519 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-820282ee-fea7-4ea5-a22c-05bcbd327dcc admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:48.753469 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-820282ee-fea7-4ea5-a22c-05bcbd327dcc admin admin] 10.222.0.39 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 846 time: 0.6778889 May 20 04:33:48.958837 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-8c3cfc14-8130-46f5-a15f-10be1c2d1def None None] Network 4155c7fb-741f-47f2-a8b6-c4b7287d1841 requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:33:49.292316 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43752) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:49.298193 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Request body: {'network': {'name': 'tempest-VolumesActionsTest-893702029-network', 'tenant_id': '31e1f0421d014269adc8341e43f05a38'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:33:49.303183 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:33:49.307508 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Usage tracker for resource:network and tenant:31e1f0421d014269adc8341e43f05a38 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:33:49.310509 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:33:49.316757 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:33:49.329223 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Made reservation on behalf of 31e1f0421d014269adc8341e43f05a38 for: {'network': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:33:49.331496 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-585852'] for network, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:49.343580 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:49.357522 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:33:49.361441 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Usage tracker for resource:security_group_rule and tenant:31e1f0421d014269adc8341e43f05a38 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:33:49.364410 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:33:49.370156 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:33:49.453046 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:49.458054 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:49.462098 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-a4c784f4-67ae-410f-818e-c36b91fd7359 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:49.492519 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 829} {{(pid=91644) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 20 04:33:49.501074 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=aa6a5604-5062-461f-aad5-bb496899151c,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:49.504609 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:49.507169 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:49.507574 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.db.segments_db [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Added segment beac99d6-305e-4168-a52b-c2fda9c77cc5 of type vlan for network a43fd0da-2837-43b6-a0c0-8d24ec70b4d4 May 20 04:33:49.533682 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8760688516720'] for network, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:49.558220 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 43762) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:49.562368 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915'] for network, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:49.563673 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:49.564254 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:49.567669 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-caa19dbb-3758-47cd-8332-e19729f78f4a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:49.667997 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] 10.222.0.39 "POST /v2.0/networks HTTP/1.1" status: 201 len: 881 time: 0.3746011 May 20 04:33:49.668932 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Pushing event updated for resources: {'Network': ['ID=a43fd0da-2837-43b6-a0c0-8d24ec70b4d4,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:49.670802 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 43768) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:49.671819 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:49.677389 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-bb719449-4087-4ac3-81b3-8b975510f98f admin admin] Request body: {'subnet': {'network_id': 'a43fd0da-2837-43b6-a0c0-8d24ec70b4d4', 'cidr': '172.20.0.0/28', 'name': 'tempest-VolumesActionsTest-893702029-subnet', 'tenant_id': '31e1f0421d014269adc8341e43f05a38', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:33:49.683197 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-bb719449-4087-4ac3-81b3-8b975510f98f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:33:49.684281 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Network a43fd0da-2837-43b6-a0c0-8d24ec70b4d4 details requested by agent ovs-agent-dv-cinder-782405-9-iscsi with host dv-cinder-782405-9-iscsi {{(pid=91645) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 20 04:33:49.687804 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-bb719449-4087-4ac3-81b3-8b975510f98f admin admin] Usage tracker for resource:subnet and tenant:31e1f0421d014269adc8341e43f05a38 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:33:49.690963 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-bb719449-4087-4ac3-81b3-8b975510f98f admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:33:49.697271 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-bb719449-4087-4ac3-81b3-8b975510f98f admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:33:49.708564 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-bb719449-4087-4ac3-81b3-8b975510f98f admin admin] Made reservation on behalf of 31e1f0421d014269adc8341e43f05a38 for: {'subnet': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:33:49.711966 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bb719449-4087-4ac3-81b3-8b975510f98f admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2172529'] for subnet, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:50.553448 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bb719449-4087-4ac3-81b3-8b975510f98f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-2050444', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2172483'] for subnet, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:50.559576 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-5565ad78-72b2-4670-8f64-475aa52aa085 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:50.618099 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-40628d21-1644-4afa-8dc7-2837ad9834b7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-8003145'] for router_interface, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:50.714685 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bb719449-4087-4ac3-81b3-8b975510f98f admin admin] Pushing event updated for resources: {'Subnet': ['ID=85781642-ef1b-44d7-bf08-65a66f8261c0,revision_number=0']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:50.722233 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-bb719449-4087-4ac3-81b3-8b975510f98f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.163s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:50.776865 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-bb719449-4087-4ac3-81b3-8b975510f98f admin admin] Checking segment: {'id': 'beac99d6-305e-4168-a52b-c2fda9c77cc5', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 829, 'network_id': 'a43fd0da-2837-43b6-a0c0-8d24ec70b4d4'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91643) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:33:50.777218 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-bb719449-4087-4ac3-81b3-8b975510f98f admin admin] Network hostable DHCP agents. Network: a43fd0da-2837-43b6-a0c0-8d24ec70b4d4, hostable agents: ['fffae25a-f450-4182-8f9e-ac29d143f6c9'], hosted agents: [] {{(pid=91643) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 20 04:33:50.869841 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-bb719449-4087-4ac3-81b3-8b975510f98f admin admin] Network a43fd0da-2837-43b6-a0c0-8d24ec70b4d4 is scheduled to be hosted by DHCP agent fffae25a-f450-4182-8f9e-ac29d143f6c9 with binding_index 1 {{(pid=91643) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 20 04:33:50.916690 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.base_scheduler [None req-bb719449-4087-4ac3-81b3-8b975510f98f admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-782405-9-iscsi, a43fd0da-2837-43b6-a0c0-8d24ec70b4d4) {{(pid=91643) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:33:50.924315 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-1a6aa57b-30a0-4542-b4f3-2e1d6e8b8d0d None None] Network a43fd0da-2837-43b6-a0c0-8d24ec70b4d4 requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:33:50.932347 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-bb719449-4087-4ac3-81b3-8b975510f98f admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:33:50.932940 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bb719449-4087-4ac3-81b3-8b975510f98f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:50.959730 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-bb719449-4087-4ac3-81b3-8b975510f98f admin admin] 10.222.0.39 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 845 time: 1.2875249 May 20 04:33:51.114959 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-df350dd1-8079-4573-9181-f4df52a8f247 None None] Network a43fd0da-2837-43b6-a0c0-8d24ec70b4d4 requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:33:51.208148 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-40628d21-1644-4afa-8dc7-2837ad9834b7 admin admin] Deleting port ac9495b7-5868-4104-85c9-3567f1350467 {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:51.208489 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-40628d21-1644-4afa-8dc7-2837ad9834b7 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:51.331344 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [None req-40628d21-1644-4afa-8dc7-2837ad9834b7 admin admin] For port ac9495b7-5868-4104-85c9-3567f1350467, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=ac9495b7-5868-4104-85c9-3567f1350467,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:33:51.331697 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-40628d21-1644-4afa-8dc7-2837ad9834b7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:51.454951 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-40628d21-1644-4afa-8dc7-2837ad9834b7 admin admin] Calling delete_port for ac9495b7-5868-4104-85c9-3567f1350467 owned by network:router_interface {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:51.612494 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-40628d21-1644-4afa-8dc7-2837ad9834b7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:51.616696 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-fb9df92b-aae9-4fb3-bbdb-51e830f4d010 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:51.661254 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-40628d21-1644-4afa-8dc7-2837ad9834b7 admin admin] Pushing event deleted for resources: {'Port': ['ID=ac9495b7-5868-4104-85c9-3567f1350467,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:51.665352 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-40628d21-1644-4afa-8dc7-2837ad9834b7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:51.938039 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-40628d21-1644-4afa-8dc7-2837ad9834b7 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-7924922', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-8054745'] for router_interface, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:52.306235 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-40628d21-1644-4afa-8dc7-2837ad9834b7 admin admin] Router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91643) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:33:52.495250 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-40628d21-1644-4afa-8dc7-2837ad9834b7 admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:33:52.504320 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-31884c01-da77-4a82-a2a9-58c52e5c5f79 None None] Sync routers for ids ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'] in dv-cinder-782405-9-iscsi {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:33:52.507755 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-40628d21-1644-4afa-8dc7-2837ad9834b7 admin admin] 10.222.0.39 "PUT /v2.0/routers/5a3f69b9-fd58-499f-bec9-75215cd7f8fc/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.9484398 May 20 04:33:52.511124 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 43786) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:52.515156 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-31884c01-da77-4a82-a2a9-58c52e5c5f79 None None] Routers {'5a3f69b9-fd58-499f-bec9-75215cd7f8fc'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:33:52.515786 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-31884c01-da77-4a82-a2a9-58c52e5c5f79 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:33:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:52.694836 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-31884c01-da77-4a82-a2a9-58c52e5c5f79 None None] Set router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:33:52.798625 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b4309ff0-c600-47ce-a0a2-6116679b26f2 admin admin] Notify callbacks [] for router, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:52.976340 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-31884c01-da77-4a82-a2a9-58c52e5c5f79 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202'}], 'enable_snat': True}, 'gw_port_id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:33:51Z', 'revision_number': 5, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port': {'id': '063b884b-9614-48ee-9ca9-0c68b3e5f35f', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:e4:9a:7c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'device_owner': 'network:router_gateway', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.202', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:21:39Z', 'updated_at': '2021-05-20T04:21:47Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:52.977921 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b4309ff0-c600-47ce-a0a2-6116679b26f2 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:52.980911 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-31884c01-da77-4a82-a2a9-58c52e5c5f79 None None] Checking router: 5a3f69b9-fd58-499f-bec9-75215cd7f8fc for host: dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:33:52.990553 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-31884c01-da77-4a82-a2a9-58c52e5c5f79 None None] Routers {'5a3f69b9-fd58-499f-bec9-75215cd7f8fc'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:33:52.991529 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-31884c01-da77-4a82-a2a9-58c52e5c5f79 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:33:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:53.051110 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-b4309ff0-c600-47ce-a0a2-6116679b26f2 admin admin] Deleting port 063b884b-9614-48ee-9ca9-0c68b3e5f35f {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:53.051556 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b4309ff0-c600-47ce-a0a2-6116679b26f2 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:53.159094 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-31884c01-da77-4a82-a2a9-58c52e5c5f79 None None] Set router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc gateway port host: None {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:33:53.173766 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [None req-b4309ff0-c600-47ce-a0a2-6116679b26f2 admin admin] For port 063b884b-9614-48ee-9ca9-0c68b3e5f35f, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=063b884b-9614-48ee-9ca9-0c68b3e5f35f,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:33:53.174202 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b4309ff0-c600-47ce-a0a2-6116679b26f2 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:53.298735 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-31884c01-da77-4a82-a2a9-58c52e5c5f79 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:33:52Z', 'revision_number': 6, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port_host': None}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:53.302922 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-31884c01-da77-4a82-a2a9-58c52e5c5f79 None None] The sync data for ids ['5a3f69b9-fd58-499f-bec9-75215cd7f8fc'] in dv-cinder-782405-9-iscsi is: [{'id': '5a3f69b9-fd58-499f-bec9-75215cd7f8fc', 'name': 'tempest-AttachVolumeShelveTestJSON-1065947157-router', 'tenant_id': '9fafdf3a29024ac588e5b24719735677', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:21:38Z', 'updated_at': '2021-05-20T04:33:52Z', 'revision_number': 6, 'project_id': '9fafdf3a29024ac588e5b24719735677', 'gw_port_host': None}] {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:33:53.308811 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-b4309ff0-c600-47ce-a0a2-6116679b26f2 admin admin] Calling delete_port for 063b884b-9614-48ee-9ca9-0c68b3e5f35f owned by network:router_gateway {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:53.451388 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b4309ff0-c600-47ce-a0a2-6116679b26f2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:53.455877 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-b44e7aff-acb4-47b4-b69f-bed9f2333fc4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:53.501559 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b4309ff0-c600-47ce-a0a2-6116679b26f2 admin admin] Pushing event deleted for resources: {'Port': ['ID=063b884b-9614-48ee-9ca9-0c68b3e5f35f,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:53.504508 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-b4309ff0-c600-47ce-a0a2-6116679b26f2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:53.592859 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b4309ff0-c600-47ce-a0a2-6116679b26f2 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-8053346'] for router_gateway, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:53.770785 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b4309ff0-c600-47ce-a0a2-6116679b26f2 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-8071641', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-260698'] for router, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:53.773154 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b4309ff0-c600-47ce-a0a2-6116679b26f2 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:53.800586 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b4309ff0-c600-47ce-a0a2-6116679b26f2 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-8056629'] for router, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:53.801230 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-b4309ff0-c600-47ce-a0a2-6116679b26f2 admin admin] Fanout notify agent at l3_agent the message router_deleted on router 5a3f69b9-fd58-499f-bec9-75215cd7f8fc {{(pid=91643) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 20 04:33:53.812868 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-b4309ff0-c600-47ce-a0a2-6116679b26f2 admin admin] Notify callbacks [] for router, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:53.813994 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-b4309ff0-c600-47ce-a0a2-6116679b26f2 admin admin] 10.222.0.39 "DELETE /v2.0/routers/5a3f69b9-fd58-499f-bec9-75215cd7f8fc HTTP/1.1" status: 204 len: 168 time: 1.3019574 May 20 04:33:53.816549 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 43802) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:53.877007 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_v2 [None req-3eaab710-5a23-428d-a66a-211fb3249c1e admin admin] Deleting subnet 203a3962-2282-4f51-ac92-c9695b5c0cfe {{(pid=91643) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 20 04:33:53.879971 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3eaab710-5a23-428d-a66a-211fb3249c1e admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:53.929290 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3eaab710-5a23-428d-a66a-211fb3249c1e admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-1472754'] for subnet, precommit_delete_associations {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:54.005224 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3eaab710-5a23-428d-a66a-211fb3249c1e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-7786687'] for subnet, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:54.123390 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 063b884b-9614-48ee-9ca9-0c68b3e5f35f no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:33:54.162355 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: 063b884b-9614-48ee-9ca9-0c68b3e5f35f {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:33:54.162991 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 063b884b-9614-48ee-9ca9-0c68b3e5f35f not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:33:54.163755 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device ac9495b7-5868-4104-85c9-3567f1350467 no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:33:54.196971 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: ac9495b7-5868-4104-85c9-3567f1350467 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:33:54.197322 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device ac9495b7-5868-4104-85c9-3567f1350467 not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:33:54.365229 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3eaab710-5a23-428d-a66a-211fb3249c1e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-7786632', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2050499', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2172490'] for subnet, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:54.369300 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:54.405429 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3eaab710-5a23-428d-a66a-211fb3249c1e admin admin] Pushing event deleted for resources: {'Subnet': ['ID=203a3962-2282-4f51-ac92-c9695b5c0cfe,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:54.407976 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-3eaab710-5a23-428d-a66a-211fb3249c1e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.039s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:54.417769 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-d1116dd2-ec98-4dd7-8032-b59f25a7e0de None None] Network 64bcea8a-6d35-406b-a823-fa1185008174 requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:33:54.418757 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3eaab710-5a23-428d-a66a-211fb3249c1e admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:54.427478 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-3eaab710-5a23-428d-a66a-211fb3249c1e admin admin] 10.222.0.39 "DELETE /v2.0/subnets/203a3962-2282-4f51-ac92-c9695b5c0cfe HTTP/1.1" status: 204 len: 168 time: 0.6077080 May 20 04:33:54.428241 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 43810) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:54.538828 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-593607'] for network, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:54.618120 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] Deleting port ca916c39-7b0c-4a56-94cc-6d171114540b {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:54.618466 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:54.672392 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-df1d3eb6-cafc-4b9c-9720-c3131933513b None None] DHCP port deletion for 64bcea8a-6d35-406b-a823-fa1185008174 request from dv-cinder-782405-9-iscsi {{(pid=91645) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 20 04:33:54.728596 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] For port ca916c39-7b0c-4a56-94cc-6d171114540b, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=ca916c39-7b0c-4a56-94cc-6d171114540b,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:33:54.729546 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:54.751229 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-df1d3eb6-cafc-4b9c-9720-c3131933513b None None] Deleting port ca916c39-7b0c-4a56-94cc-6d171114540b {{(pid=91645) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:33:54.751621 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-df1d3eb6-cafc-4b9c-9720-c3131933513b None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:54.856242 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-df1d3eb6-cafc-4b9c-9720-c3131933513b None None] For port ca916c39-7b0c-4a56-94cc-6d171114540b, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=ca916c39-7b0c-4a56-94cc-6d171114540b,segment=NetworkSegment(9d1b8515-c35d-48e6-86ad-4eff412e924a),segment_id=9d1b8515-c35d-48e6-86ad-4eff412e924a)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:33:54.856614 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-df1d3eb6-cafc-4b9c-9720-c3131933513b None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:54.858763 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] Calling delete_port for ca916c39-7b0c-4a56-94cc-6d171114540b owned by network:dhcp {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:54.905936 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:54.911194 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-84078db7-6f3a-45c8-ad1e-bc5b2c48a8d9 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:54.958122 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] Pushing event deleted for resources: {'Port': ['ID=ca916c39-7b0c-4a56-94cc-6d171114540b,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:54.960195 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] No DVR routers for this DVR port ca916c39-7b0c-4a56-94cc-6d171114540b on host dv-cinder-782405-9-iscsi {{(pid=91643) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:33:54.961741 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:54.979163 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-df1d3eb6-cafc-4b9c-9720-c3131933513b None None] Calling delete_port for ca916c39-7b0c-4a56-94cc-6d171114540b owned by network:dhcp {{(pid=91645) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:33:55.022494 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-df1d3eb6-cafc-4b9c-9720-c3131933513b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716'] for port, after_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:55.026079 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-e8173c90-339a-46e1-b2b3-da5d98731dcd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:55.076472 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-7789108', 'neutron.services.segments.db._delete_segments_for_network-8760688958082'] for network, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:55.091067 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['64bcea8a-6d35-406b-a823-fa1185008174']}} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:55.104918 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '9d1b8515-c35d-48e6-86ad-4eff412e924a') {'for_net_delete': True} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:55.105198 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '9d1b8515-c35d-48e6-86ad-4eff412e924a') {} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:33:55.109246 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-df1d3eb6-cafc-4b9c-9720-c3131933513b None None] Pushing event deleted for resources: {'Port': ['ID=ca916c39-7b0c-4a56-94cc-6d171114540b,revision_number=None']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:55.111648 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-df1d3eb6-cafc-4b9c-9720-c3131933513b None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:55.113952 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-df1d3eb6-cafc-4b9c-9720-c3131933513b None None] No DVR routers for this DVR port ca916c39-7b0c-4a56-94cc-6d171114540b on host dv-cinder-782405-9-iscsi {{(pid=91645) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:33:55.117624 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8760688968331', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-1472741'] for segment, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:55.134813 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:55.141000 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] Releasing vlan 847 on physical network physnet1 to pool {{(pid=91643) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 20 04:33:55.141435 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2070625'] for segment, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: Traceback (most recent call last): May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: timer() May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: cb(*args, **kw) May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: return func(*args, **kwargs) May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: self._notify() May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: self.callback(batched_events) May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: event.method(event) May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: aggregate_id = self._get_aggregate_id(event.segment_id) May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: aggregate_uuid = self.p_client.list_aggregates( May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: return f(self, *a, **k) May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: return self._get(url).json() May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: return self._client.get(url, endpoint_filter=self._ks_filter, May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: return self.request(url, 'GET', **kwargs) May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: base_url = self.get_endpoint(auth, allow=allow, May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: auth = self._auth_required(auth, 'determine endpoint URL') May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 20 04:33:55.154575 dv-cinder-782405-9-iscsi neutron-server[91643]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 20 04:33:55.236034 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-7789117', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-8053321'] for network, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:55.243521 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:55.246972 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:55.272856 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] Pushing event deleted for resources: {'Network': ['ID=64bcea8a-6d35-406b-a823-fa1185008174,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:55.275351 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:55.287438 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-bf798e9e-32da-4a0f-9069-c1e6ae423bec admin admin] 10.222.0.39 "DELETE /v2.0/networks/64bcea8a-6d35-406b-a823-fa1185008174 HTTP/1.1" status: 204 len: 168 time: 0.8561685 May 20 04:33:55.410241 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43820) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:55.459502 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-1f7b1386-22ed-4bb3-966d-78f2e88d42d2 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=9fafdf3a29024ac588e5b24719735677&name=default HTTP/1.1" status: 200 len: 2975 time: 0.0479124 May 20 04:33:55.462620 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 43824) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:33:55.520294 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-08e599fb-d72f-4d58-b203-608b8f8df392 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:55.546570 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-08e599fb-d72f-4d58-b203-608b8f8df392 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:55.604727 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-08e599fb-d72f-4d58-b203-608b8f8df392 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:55.608559 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-eb911ea4-f94f-47df-a048-51b3f0dbfea8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:33:55.622504 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-08e599fb-d72f-4d58-b203-608b8f8df392 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=4d590d22-d6da-4a07-99f3-1768191cb1d6,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:33:55.626301 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-08e599fb-d72f-4d58-b203-608b8f8df392 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:33:55.630561 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-08e599fb-d72f-4d58-b203-608b8f8df392 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:33:55.631694 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-08e599fb-d72f-4d58-b203-608b8f8df392 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/4d590d22-d6da-4a07-99f3-1768191cb1d6 HTTP/1.1" status: 204 len: 168 time: 0.1682427 May 20 04:33:56.139583 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device ca916c39-7b0c-4a56-94cc-6d171114540b no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:33:56.180457 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: ca916c39-7b0c-4a56-94cc-6d171114540b {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:33:56.180805 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device ca916c39-7b0c-4a56-94cc-6d171114540b not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:34:05.679206 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:11.873705 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 44070) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:11.933340 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_v2 [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] Deleting subnet e23e84bb-81f2-40ad-8891-cdc124efa6cd {{(pid=91643) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 20 04:34:11.936502 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:11.989519 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-1472754'] for subnet, precommit_delete_associations {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:12.044709 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-7786687'] for subnet, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:12.401603 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-7786632', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2050499', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2172490'] for subnet, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:12.407114 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:12.443034 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] Pushing event deleted for resources: {'Subnet': ['ID=e23e84bb-81f2-40ad-8891-cdc124efa6cd,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:12.447956 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:12.454864 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:12.455308 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:12.455651 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:12.456018 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:12.456378 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:12.456740 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:12.457100 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:12.457452 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:12.457808 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:12.458167 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:12.458521 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:12.458872 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:12.462400 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:12.465167 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-702c8466-62b7-47a7-a8ad-1b2711bf47ae admin admin] 10.222.0.39 "DELETE /v2.0/subnets/e23e84bb-81f2-40ad-8891-cdc124efa6cd HTTP/1.1" status: 204 len: 168 time: 0.5904584 May 20 04:34:12.466737 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 44086) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:12.575991 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f79aaf39-aefc-4640-aefc-c58e4dedcd6f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-593607'] for network, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:12.621378 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-bafc8bd4-6f2c-458f-9bde-01e1d7c7f5ea None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:12.703833 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f79aaf39-aefc-4640-aefc-c58e4dedcd6f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-7789108', 'neutron.services.segments.db._delete_segments_for_network-8760688958082'] for network, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:12.716577 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-f79aaf39-aefc-4640-aefc-c58e4dedcd6f admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['4155c7fb-741f-47f2-a8b6-c4b7287d1841']}} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:34:12.729956 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-f79aaf39-aefc-4640-aefc-c58e4dedcd6f admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '7ae61a20-156c-49d5-97f4-da7391c99834') {'for_net_delete': True} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:34:12.730253 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-f79aaf39-aefc-4640-aefc-c58e4dedcd6f admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '7ae61a20-156c-49d5-97f4-da7391c99834') {} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:34:12.741497 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f79aaf39-aefc-4640-aefc-c58e4dedcd6f admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8760688968331', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-1472741'] for segment, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:12.757330 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f79aaf39-aefc-4640-aefc-c58e4dedcd6f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:12.765462 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-f79aaf39-aefc-4640-aefc-c58e4dedcd6f admin admin] Releasing vlan 845 on physical network physnet1 to pool {{(pid=91643) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 20 04:34:12.765976 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f79aaf39-aefc-4640-aefc-c58e4dedcd6f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2070625'] for segment, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: Traceback (most recent call last): May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: timer() May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: cb(*args, **kw) May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: return func(*args, **kwargs) May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: self._notify() May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: self.callback(batched_events) May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: event.method(event) May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: aggregate_id = self._get_aggregate_id(event.segment_id) May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: aggregate_uuid = self.p_client.list_aggregates( May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: return f(self, *a, **k) May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: return self._get(url).json() May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: return self._client.get(url, endpoint_filter=self._ks_filter, May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: return self.request(url, 'GET', **kwargs) May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: base_url = self.get_endpoint(auth, allow=allow, May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: auth = self._auth_required(auth, 'determine endpoint URL') May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 20 04:34:12.781080 dv-cinder-782405-9-iscsi neutron-server[91643]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 20 04:34:12.866329 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f79aaf39-aefc-4640-aefc-c58e4dedcd6f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-7789117', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-8053321'] for network, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:12.870290 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:12.883002 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f79aaf39-aefc-4640-aefc-c58e4dedcd6f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:12.891840 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f79aaf39-aefc-4640-aefc-c58e4dedcd6f admin admin] Pushing event deleted for resources: {'Network': ['ID=4155c7fb-741f-47f2-a8b6-c4b7287d1841,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:12.894759 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-f79aaf39-aefc-4640-aefc-c58e4dedcd6f admin admin] 10.222.0.39 "DELETE /v2.0/networks/4155c7fb-741f-47f2-a8b6-c4b7287d1841 HTTP/1.1" status: 204 len: 168 time: 0.4263065 May 20 04:34:12.895565 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-f79aaf39-aefc-4640-aefc-c58e4dedcd6f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.025s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:12.897288 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 44090) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:12.956035 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_v2 [None req-aa9852cd-4e08-4345-92c2-7639d43de68d admin admin] Deleting subnet 85781642-ef1b-44d7-bf08-65a66f8261c0 {{(pid=91643) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 20 04:34:12.959354 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-aa9852cd-4e08-4345-92c2-7639d43de68d admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:13.012578 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-aa9852cd-4e08-4345-92c2-7639d43de68d admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-1472754'] for subnet, precommit_delete_associations {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:13.069498 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-aa9852cd-4e08-4345-92c2-7639d43de68d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-7786687'] for subnet, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:13.321722 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-aa9852cd-4e08-4345-92c2-7639d43de68d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-7786632', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2050499', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2172490'] for subnet, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:13.326873 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-ac5c230b-aa49-48f8-96ae-fc88dd6d809a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:13.360502 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aa9852cd-4e08-4345-92c2-7639d43de68d admin admin] Pushing event deleted for resources: {'Subnet': ['ID=85781642-ef1b-44d7-bf08-65a66f8261c0,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:13.363559 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-aa9852cd-4e08-4345-92c2-7639d43de68d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.037s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:13.364550 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-aa9852cd-4e08-4345-92c2-7639d43de68d admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:13.370749 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-aa9852cd-4e08-4345-92c2-7639d43de68d admin admin] 10.222.0.39 "DELETE /v2.0/subnets/85781642-ef1b-44d7-bf08-65a66f8261c0 HTTP/1.1" status: 204 len: 168 time: 0.4725044 May 20 04:34:13.373335 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 44100) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:13.480259 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d36cc6ca-c291-4989-b016-be779d3a68a4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-593607'] for network, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:13.609557 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d36cc6ca-c291-4989-b016-be779d3a68a4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-7789108', 'neutron.services.segments.db._delete_segments_for_network-8760688958082'] for network, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:13.623265 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-d36cc6ca-c291-4989-b016-be779d3a68a4 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['a43fd0da-2837-43b6-a0c0-8d24ec70b4d4']}} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:34:13.637190 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-d36cc6ca-c291-4989-b016-be779d3a68a4 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'beac99d6-305e-4168-a52b-c2fda9c77cc5') {'for_net_delete': True} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:34:13.637551 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-d36cc6ca-c291-4989-b016-be779d3a68a4 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'beac99d6-305e-4168-a52b-c2fda9c77cc5') {} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:34:13.650853 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d36cc6ca-c291-4989-b016-be779d3a68a4 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8760688968331', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-1472741'] for segment, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:13.670956 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d36cc6ca-c291-4989-b016-be779d3a68a4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:13.678195 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-d36cc6ca-c291-4989-b016-be779d3a68a4 admin admin] Releasing vlan 829 on physical network physnet1 to pool {{(pid=91643) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 20 04:34:13.678578 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d36cc6ca-c291-4989-b016-be779d3a68a4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2070625'] for segment, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: Traceback (most recent call last): May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: timer() May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: cb(*args, **kw) May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: return func(*args, **kwargs) May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: self._notify() May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: self.callback(batched_events) May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: event.method(event) May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: aggregate_id = self._get_aggregate_id(event.segment_id) May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: aggregate_uuid = self.p_client.list_aggregates( May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: return f(self, *a, **k) May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: return self._get(url).json() May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: return self._client.get(url, endpoint_filter=self._ks_filter, May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: return self.request(url, 'GET', **kwargs) May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: base_url = self.get_endpoint(auth, allow=allow, May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: auth = self._auth_required(auth, 'determine endpoint URL') May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 20 04:34:13.696152 dv-cinder-782405-9-iscsi neutron-server[91643]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 20 04:34:13.770780 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d36cc6ca-c291-4989-b016-be779d3a68a4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-7789117', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-8053321'] for network, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:13.774410 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:13.789499 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-d36cc6ca-c291-4989-b016-be779d3a68a4 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:13.796536 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d36cc6ca-c291-4989-b016-be779d3a68a4 admin admin] Pushing event deleted for resources: {'Network': ['ID=a43fd0da-2837-43b6-a0c0-8d24ec70b4d4,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:13.799373 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-d36cc6ca-c291-4989-b016-be779d3a68a4 admin admin] 10.222.0.39 "DELETE /v2.0/networks/a43fd0da-2837-43b6-a0c0-8d24ec70b4d4 HTTP/1.1" status: 204 len: 168 time: 0.4251101 May 20 04:34:13.800339 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-d36cc6ca-c291-4989-b016-be779d3a68a4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:13.979908 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 44114) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:14.036100 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-31330b40-46a2-43a6-96a0-b814641b7181 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=bc0ec8722e724a239e6514c8a40b6684&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0547869 May 20 04:34:14.038313 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 44118) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:14.105265 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a71ee8a8-570e-483e-9747-b35fd986a35e admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:14.131196 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a71ee8a8-570e-483e-9747-b35fd986a35e admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:14.185766 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a71ee8a8-570e-483e-9747-b35fd986a35e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:14.189548 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-a331b8d7-b2b3-4e9d-b63a-65c8c6f8b97d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:14.200870 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a71ee8a8-570e-483e-9747-b35fd986a35e admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=24943297-64ff-4c69-b9ca-ea881f47715e,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:14.204388 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-a71ee8a8-570e-483e-9747-b35fd986a35e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:14.208130 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a71ee8a8-570e-483e-9747-b35fd986a35e admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:14.209196 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-a71ee8a8-570e-483e-9747-b35fd986a35e admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/24943297-64ff-4c69-b9ca-ea881f47715e HTTP/1.1" status: 204 len: 168 time: 0.1698062 May 20 04:34:14.572764 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 44128) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:14.624605 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-8663999f-f92b-4caf-9287-ffc170788c30 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=31e1f0421d014269adc8341e43f05a38&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0506060 May 20 04:34:14.628036 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 44132) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:14.689683 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-443aa9cc-7ac1-4c83-8d00-7b85c250efdb admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:14.716658 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-443aa9cc-7ac1-4c83-8d00-7b85c250efdb admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:14.771672 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-443aa9cc-7ac1-4c83-8d00-7b85c250efdb admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:14.775366 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-2afec35f-c57d-4b64-9471-452a8fe09c8c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:14.788067 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-443aa9cc-7ac1-4c83-8d00-7b85c250efdb admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=aa6a5604-5062-461f-aad5-bb496899151c,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:14.792389 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-443aa9cc-7ac1-4c83-8d00-7b85c250efdb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:14.794038 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-443aa9cc-7ac1-4c83-8d00-7b85c250efdb admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:14.798406 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-443aa9cc-7ac1-4c83-8d00-7b85c250efdb admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/aa6a5604-5062-461f-aad5-bb496899151c HTTP/1.1" status: 204 len: 168 time: 0.1656513 May 20 04:34:14.979918 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:15.111473 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e18ace08-97e7-448b-85bf-2aefe6706fae None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:16.235191 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:17.871565 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:23.024113 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:34:35.033723 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 44366) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:35.151930 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2d3a58cc-3635-4c0b-9df6-b7a7b3160f14 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:35.166435 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-2d3a58cc-3635-4c0b-9df6-b7a7b3160f14 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:34:35.170593 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-2d3a58cc-3635-4c0b-9df6-b7a7b3160f14 admin admin] Usage tracker for resource:security_group_rule and tenant:be336a125d81407a9b6a13dfcc9ac73e is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:34:35.174296 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-2d3a58cc-3635-4c0b-9df6-b7a7b3160f14 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:34:35.180416 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-2d3a58cc-3635-4c0b-9df6-b7a7b3160f14 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:34:35.266727 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2d3a58cc-3635-4c0b-9df6-b7a7b3160f14 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:35.271071 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-2d3a58cc-3635-4c0b-9df6-b7a7b3160f14 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:35.281013 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-e3a4a585-6853-4707-a15f-7c8c424da2e1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:35.322685 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-2d3a58cc-3635-4c0b-9df6-b7a7b3160f14 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=be336a125d81407a9b6a13dfcc9ac73e&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2854178 May 20 04:34:35.322685 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2d3a58cc-3635-4c0b-9df6-b7a7b3160f14 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=3312b0e8-dfa0-4e9b-a805-d223bee82f08,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:35.323379 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 44384) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:35.324593 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-2d3a58cc-3635-4c0b-9df6-b7a7b3160f14 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:35.390310 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0402b0dd-e7c5-4352-9c9c-c8d843de3436 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:35.416416 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0402b0dd-e7c5-4352-9c9c-c8d843de3436 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:35.470606 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0402b0dd-e7c5-4352-9c9c-c8d843de3436 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:35.477308 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-b74cab4e-b81c-4278-8976-ef7657eef473 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:35.488724 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0402b0dd-e7c5-4352-9c9c-c8d843de3436 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=3312b0e8-dfa0-4e9b-a805-d223bee82f08,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:35.498899 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0402b0dd-e7c5-4352-9c9c-c8d843de3436 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:35.499300 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0402b0dd-e7c5-4352-9c9c-c8d843de3436 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:35.499666 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0402b0dd-e7c5-4352-9c9c-c8d843de3436 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:35.500026 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0402b0dd-e7c5-4352-9c9c-c8d843de3436 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:35.500384 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0402b0dd-e7c5-4352-9c9c-c8d843de3436 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:35.500739 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0402b0dd-e7c5-4352-9c9c-c8d843de3436 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:35.501088 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0402b0dd-e7c5-4352-9c9c-c8d843de3436 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:35.501435 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0402b0dd-e7c5-4352-9c9c-c8d843de3436 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:35.501810 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0402b0dd-e7c5-4352-9c9c-c8d843de3436 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:35.502170 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0402b0dd-e7c5-4352-9c9c-c8d843de3436 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:35.502515 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0402b0dd-e7c5-4352-9c9c-c8d843de3436 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:35.502904 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0402b0dd-e7c5-4352-9c9c-c8d843de3436 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:34:35.506023 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-0402b0dd-e7c5-4352-9c9c-c8d843de3436 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:35.507177 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-0402b0dd-e7c5-4352-9c9c-c8d843de3436 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/3312b0e8-dfa0-4e9b-a805-d223bee82f08 HTTP/1.1" status: 204 len: 168 time: 0.1825898 May 20 04:34:35.508125 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-0402b0dd-e7c5-4352-9c9c-c8d843de3436 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.031s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:35.692051 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:35.805280 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 44394) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:35.824513 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-31554eda-ea2a-4b9d-931e-8a515df61305 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:35.838962 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-31554eda-ea2a-4b9d-931e-8a515df61305 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:34:35.843135 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-31554eda-ea2a-4b9d-931e-8a515df61305 admin admin] Usage tracker for resource:security_group_rule and tenant:75af2dd97d0d4fa198be37b837600d95 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:34:35.845928 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-31554eda-ea2a-4b9d-931e-8a515df61305 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:34:35.851602 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-31554eda-ea2a-4b9d-931e-8a515df61305 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:34:35.930346 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-31554eda-ea2a-4b9d-931e-8a515df61305 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:35.936122 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-31554eda-ea2a-4b9d-931e-8a515df61305 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:35.944829 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-4fcc8c00-8f3c-4262-9d97-c367b7298369 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:35.986183 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-31554eda-ea2a-4b9d-931e-8a515df61305 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=75af2dd97d0d4fa198be37b837600d95&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1797791 May 20 04:34:35.987054 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-31554eda-ea2a-4b9d-931e-8a515df61305 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=06dd5787-3525-4220-945a-d197c9927514,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:35.989385 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 44398) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:35.990153 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-31554eda-ea2a-4b9d-931e-8a515df61305 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:36.061618 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6e9969ec-7394-4283-a647-a97f781e13b9 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:36.088675 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6e9969ec-7394-4283-a647-a97f781e13b9 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:36.141776 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6e9969ec-7394-4283-a647-a97f781e13b9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:36.145567 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-f75ec2a5-15b3-4fa5-aaf0-69ad1b54c817 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:36.158341 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6e9969ec-7394-4283-a647-a97f781e13b9 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=06dd5787-3525-4220-945a-d197c9927514,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:36.162180 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-6e9969ec-7394-4283-a647-a97f781e13b9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:36.167748 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6e9969ec-7394-4283-a647-a97f781e13b9 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:36.168838 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-6e9969ec-7394-4283-a647-a97f781e13b9 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/06dd5787-3525-4220-945a-d197c9927514 HTTP/1.1" status: 204 len: 168 time: 0.1784372 May 20 04:34:42.631238 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-40faf282-5544-4635-bd68-03a97172fab5 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:43.668628 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 44488) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:43.786738 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-47b120ac-3ddc-4f02-8a45-bd6eb30d88fd admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:43.801422 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-47b120ac-3ddc-4f02-8a45-bd6eb30d88fd admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:34:43.805635 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-47b120ac-3ddc-4f02-8a45-bd6eb30d88fd admin admin] Usage tracker for resource:security_group_rule and tenant:b1c075bb2cae4e349f76f3225465101d is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:34:43.808704 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-47b120ac-3ddc-4f02-8a45-bd6eb30d88fd admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:34:43.814684 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-47b120ac-3ddc-4f02-8a45-bd6eb30d88fd admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:34:43.901821 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-47b120ac-3ddc-4f02-8a45-bd6eb30d88fd admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:43.906976 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-47b120ac-3ddc-4f02-8a45-bd6eb30d88fd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:43.915552 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-20542b25-396f-460c-ab64-9d8ed132d3c5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:43.953701 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-47b120ac-3ddc-4f02-8a45-bd6eb30d88fd admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=b1c075bb2cae4e349f76f3225465101d&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2840126 May 20 04:34:43.955317 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-47b120ac-3ddc-4f02-8a45-bd6eb30d88fd admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=b4252baf-9697-4eb5-8ae4-6cf08afe622a,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:43.956459 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 44504) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:43.958375 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-47b120ac-3ddc-4f02-8a45-bd6eb30d88fd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:44.017414 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f93df6e9-fc9c-4744-abf7-01b24bb1b3d8 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:44.043012 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f93df6e9-fc9c-4744-abf7-01b24bb1b3d8 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:44.101000 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f93df6e9-fc9c-4744-abf7-01b24bb1b3d8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:44.106068 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-707e6e81-8983-4099-8011-96084158973e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:44.204954 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f93df6e9-fc9c-4744-abf7-01b24bb1b3d8 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=b4252baf-9697-4eb5-8ae4-6cf08afe622a,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:44.207747 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-f93df6e9-fc9c-4744-abf7-01b24bb1b3d8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.102s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:44.209821 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f93df6e9-fc9c-4744-abf7-01b24bb1b3d8 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:44.210921 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-f93df6e9-fc9c-4744-abf7-01b24bb1b3d8 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/b4252baf-9697-4eb5-8ae4-6cf08afe622a HTTP/1.1" status: 204 len: 168 time: 0.2533369 May 20 04:34:44.976039 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:45.121979 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-d6cc35a4-0f4d-4d42-b1b8-210063af68cf None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:46.242005 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:46.750013 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 44578) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:47.049470 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-7746135d-7f3f-4606-8224-6167eaee66cf tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Deleting port 60843dfc-2df1-42ea-a3d7-059c71b23f9c {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:34:47.050222 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7746135d-7f3f-4606-8224-6167eaee66cf tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:47.197843 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7746135d-7f3f-4606-8224-6167eaee66cf tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:47.198176 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7746135d-7f3f-4606-8224-6167eaee66cf tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:47.317925 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-7746135d-7f3f-4606-8224-6167eaee66cf tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Calling delete_port for 60843dfc-2df1-42ea-a3d7-059c71b23f9c owned by network:floatingip {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:34:47.347436 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 44602) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:47.448891 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7746135d-7f3f-4606-8224-6167eaee66cf tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:47.453854 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-c867a753-80c5-4b73-ae2b-0cd17caa9bd7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:47.465570 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8d090719-4bab-4d4c-8e33-9a9bbba18c5e admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:47.480072 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-8d090719-4bab-4d4c-8e33-9a9bbba18c5e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:34:47.484097 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-8d090719-4bab-4d4c-8e33-9a9bbba18c5e admin admin] Usage tracker for resource:security_group_rule and tenant:2a9fcbf5780449c8aac6c58ad8992d83 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:34:47.486949 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-8d090719-4bab-4d4c-8e33-9a9bbba18c5e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:34:47.492651 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-8d090719-4bab-4d4c-8e33-9a9bbba18c5e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:34:47.496104 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7746135d-7f3f-4606-8224-6167eaee66cf tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event deleted for resources: {'Port': ['ID=60843dfc-2df1-42ea-a3d7-059c71b23f9c,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:47.499022 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-7746135d-7f3f-4606-8224-6167eaee66cf tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:47.574880 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8d090719-4bab-4d4c-8e33-9a9bbba18c5e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:47.579877 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8d090719-4bab-4d4c-8e33-9a9bbba18c5e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:47.592173 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-42d44108-3825-4080-a8ff-c08242d17cae admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:47.593311 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7746135d-7f3f-4606-8224-6167eaee66cf tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for floatingip, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:47.593815 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.l3_db [None req-7746135d-7f3f-4606-8224-6167eaee66cf tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Floating IP e1254a73-238b-496d-bda5-e60da4e8c2d4 disassociated (deleted). External IP: 172.27.0.163, port: fb2044c2-038a-473c-968f-29ae50d3c1a4. May 20 04:34:47.630384 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-8d090719-4bab-4d4c-8e33-9a9bbba18c5e admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=2a9fcbf5780449c8aac6c58ad8992d83&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2819090 May 20 04:34:47.632139 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8d090719-4bab-4d4c-8e33-9a9bbba18c5e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=a45f164c-788b-4a25-8fc9-c3456e93330c,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:47.634357 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 44608) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:47.638099 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-8d090719-4bab-4d4c-8e33-9a9bbba18c5e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:47.696064 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c80039e3-7ece-47f2-94b8-a49167903484 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:47.721908 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c80039e3-7ece-47f2-94b8-a49167903484 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:47.775272 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c80039e3-7ece-47f2-94b8-a49167903484 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:47.780242 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-778a8260-046a-4610-b83c-64fb5b0e408a tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:47.794261 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c80039e3-7ece-47f2-94b8-a49167903484 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=a45f164c-788b-4a25-8fc9-c3456e93330c,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:47.797797 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-c80039e3-7ece-47f2-94b8-a49167903484 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:47.810315 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-c80039e3-7ece-47f2-94b8-a49167903484 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:47.814012 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-c80039e3-7ece-47f2-94b8-a49167903484 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/a45f164c-788b-4a25-8fc9-c3456e93330c HTTP/1.1" status: 204 len: 168 time: 0.1754973 May 20 04:34:47.872299 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:47.949459 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-7746135d-7f3f-4606-8224-6167eaee66cf tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Router 4969b258-f606-4def-b666-09764c475749 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91643) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:34:48.096485 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 44616) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:48.114882 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-380fe57a-7917-4661-991e-c37ef1403504 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:48.132669 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-380fe57a-7917-4661-991e-c37ef1403504 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:34:48.136824 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-380fe57a-7917-4661-991e-c37ef1403504 admin admin] Usage tracker for resource:security_group_rule and tenant:98b52c3066044202b3bf7864ebe0daa9 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:34:48.139842 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-380fe57a-7917-4661-991e-c37ef1403504 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:34:48.145767 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-380fe57a-7917-4661-991e-c37ef1403504 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:34:48.240719 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-380fe57a-7917-4661-991e-c37ef1403504 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:48.245596 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-380fe57a-7917-4661-991e-c37ef1403504 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:48.249254 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-7746135d-7f3f-4606-8224-6167eaee66cf tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:34:48.263637 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9e6439f1-e43b-454c-b8e0-e28cea446f0a None None] Sync routers for ids ['4969b258-f606-4def-b666-09764c475749'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:34:48.267237 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-7746135d-7f3f-4606-8224-6167eaee66cf tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:48.274779 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9e6439f1-e43b-454c-b8e0-e28cea446f0a None None] Routers {'4969b258-f606-4def-b666-09764c475749'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:34:48.275571 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-bb717663-9293-41ed-b5ec-3d358fca98da admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:48.275728 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-9e6439f1-e43b-454c-b8e0-e28cea446f0a None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:34:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['4969b258-f606-4def-b666-09764c475749'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:34:48.286549 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-380fe57a-7917-4661-991e-c37ef1403504 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=98b52c3066044202b3bf7864ebe0daa9&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1866729 May 20 04:34:48.286775 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 44624) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:48.326998 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-7746135d-7f3f-4606-8224-6167eaee66cf tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "DELETE /v2.0/floatingips/e1254a73-238b-496d-bda5-e60da4e8c2d4 HTTP/1.1" status: 204 len: 168 time: 1.5755744 May 20 04:34:48.327833 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '7cc291a5-7b2b-48ad-af24-f0fbbbf92685', 'tag': 'fb2044c2-038a-473c-968f-29ae50d3c1a4'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:34:48.362638 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-380fe57a-7917-4661-991e-c37ef1403504 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=8c0d24b0-5209-468c-9c8b-96a73faa67ea,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:48.378401 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-380fe57a-7917-4661-991e-c37ef1403504 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.103s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:48.397266 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-2c7888a6-2c7d-4ba1-8847-d445ff897ccf admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:48.401702 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '7cc291a5-7b2b-48ad-af24-f0fbbbf92685', 'tag': 'fb2044c2-038a-473c-968f-29ae50d3c1a4', 'status': 'completed', 'code': 200} May 20 04:34:48.425669 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-2c7888a6-2c7d-4ba1-8847-d445ff897ccf admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:48.481442 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-2c7888a6-2c7d-4ba1-8847-d445ff897ccf admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:48.489062 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-c2001ae1-8c48-4ab2-80ea-c7b69ccc9900 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:48.502823 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c7888a6-2c7d-4ba1-8847-d445ff897ccf admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=8c0d24b0-5209-468c-9c8b-96a73faa67ea,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:48.507931 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-2c7888a6-2c7d-4ba1-8847-d445ff897ccf admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:48.522454 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-2c7888a6-2c7d-4ba1-8847-d445ff897ccf admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:48.526891 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-2c7888a6-2c7d-4ba1-8847-d445ff897ccf admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/8c0d24b0-5209-468c-9c8b-96a73faa67ea HTTP/1.1" status: 204 len: 168 time: 0.2389169 May 20 04:34:48.551926 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-80f4c630-a822-47c9-9f78-a33150f33261 req-d9b17af0-d591-4e49-b81a-50b54b932066 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:34:48.553962 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-80f4c630-a822-47c9-9f78-a33150f33261 req-d9b17af0-d591-4e49-b81a-50b54b932066 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=0e3baa0ea24d42f3ab59e40845b350fd&device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 1098 time: 0.1247885 May 20 04:34:48.610337 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-9e6439f1-e43b-454c-b8e0-e28cea446f0a None None] Set router 4969b258-f606-4def-b666-09764c475749 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:34:48.800794 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-80f4c630-a822-47c9-9f78-a33150f33261 req-758ddcc1-f9cf-4b31-883b-e6e644edb659 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:34:48.803098 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-80f4c630-a822-47c9-9f78-a33150f33261 req-758ddcc1-f9cf-4b31-883b-e6e644edb659 service neutron] 10.222.0.74 "GET /v2.0/networks?id=14c537db-01aa-4360-a348-47f94c75e03b HTTP/1.1" status: 200 len: 911 time: 0.2376366 May 20 04:34:48.900820 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c17572d0-d371-431e-a494-3c989cafeb5e req-49006087-1b41-48c3-bbdc-2a13e6d635a5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:34:48.901547 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-c17572d0-d371-431e-a494-3c989cafeb5e req-49006087-1b41-48c3-bbdc-2a13e6d635a5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Synchronizing usage tracker for tenant:0e3baa0ea24d42f3ab59e40845b350fd on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:34:48.907462 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-80f4c630-a822-47c9-9f78-a33150f33261 req-9fa3aca4-55c6-4716-8e75-6b83648750a6 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=fb2044c2-038a-473c-968f-29ae50d3c1a4 HTTP/1.1" status: 200 len: 217 time: 0.0924199 May 20 04:34:48.954580 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-c17572d0-d371-431e-a494-3c989cafeb5e req-49006087-1b41-48c3-bbdc-2a13e6d635a5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Unset dirty status for tenant:0e3baa0ea24d42f3ab59e40845b350fd on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:34:48.963229 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c17572d0-d371-431e-a494-3c989cafeb5e req-49006087-1b41-48c3-bbdc-2a13e6d635a5 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.2685869 May 20 04:34:49.032837 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-80f4c630-a822-47c9-9f78-a33150f33261 req-be455a88-b1f1-4f08-8e5b-4abcb059833e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:34:49.034494 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-80f4c630-a822-47c9-9f78-a33150f33261 req-be455a88-b1f1-4f08-8e5b-4abcb059833e service neutron] 10.222.0.74 "GET /v2.0/subnets?id=cdea475c-2929-4ae4-9271-87cc2fb9a378 HTTP/1.1" status: 200 len: 849 time: 0.1176729 May 20 04:34:49.049893 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c17572d0-d371-431e-a494-3c989cafeb5e req-94e72225-baae-420f-884c-efe415c880aa tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0857484 May 20 04:34:49.056425 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-9e6439f1-e43b-454c-b8e0-e28cea446f0a None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'f7e7609b-a65e-451e-8db6-b6e1bcbaa2e9', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'floating_ip_address': '172.27.0.168', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '4969b258-f606-4def-b666-09764c475749', 'port_id': 'ca396585-cdb5-4a50-a4c9-d5954043208e', 'fixed_ip_address': '172.20.0.5', 'status': 'ACTIVE', 'standard_attr_id': 569, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'mac_address': 'fa:16:3e:37:08:e4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:29:15Z', 'updated_at': '2021-05-20T04:29:18Z', 'revision_number': 1, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:50Z', 'revision_number': 5, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:34:49.061789 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9e6439f1-e43b-454c-b8e0-e28cea446f0a None None] Checking router: 4969b258-f606-4def-b666-09764c475749 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:34:49.073427 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9e6439f1-e43b-454c-b8e0-e28cea446f0a None None] Routers {'4969b258-f606-4def-b666-09764c475749'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:34:49.074739 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-9e6439f1-e43b-454c-b8e0-e28cea446f0a None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:34:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['4969b258-f606-4def-b666-09764c475749'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:34:49.145246 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-80f4c630-a822-47c9-9f78-a33150f33261 req-99878f24-6d23-4f41-8da9-bc1c5b32c97e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:34:49.146958 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-80f4c630-a822-47c9-9f78-a33150f33261 req-99878f24-6d23-4f41-8da9-bc1c5b32c97e service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=14c537db-01aa-4360-a348-47f94c75e03b&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0959387 May 20 04:34:49.268685 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-9e6439f1-e43b-454c-b8e0-e28cea446f0a None None] Set router 4969b258-f606-4def-b666-09764c475749 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:34:49.293054 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-80f4c630-a822-47c9-9f78-a33150f33261 req-ef3ce958-8c90-412e-9eef-5573be502de0 service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1374826 May 20 04:34:49.484845 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-80f4c630-a822-47c9-9f78-a33150f33261 req-bbcd7781-0d76-4a8c-8c6e-c0464b631681 service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1844833 May 20 04:34:49.645259 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-9e6439f1-e43b-454c-b8e0-e28cea446f0a None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'f7e7609b-a65e-451e-8db6-b6e1bcbaa2e9', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'floating_ip_address': '172.27.0.168', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '4969b258-f606-4def-b666-09764c475749', 'port_id': 'ca396585-cdb5-4a50-a4c9-d5954043208e', 'fixed_ip_address': '172.20.0.5', 'status': 'ACTIVE', 'standard_attr_id': 569, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'mac_address': 'fa:16:3e:37:08:e4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:29:15Z', 'updated_at': '2021-05-20T04:29:18Z', 'revision_number': 1, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:50Z', 'revision_number': 5, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:34:49.649967 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9e6439f1-e43b-454c-b8e0-e28cea446f0a None None] The sync data for ids ['4969b258-f606-4def-b666-09764c475749'] in dv-cinder-782405-9-iscsi is: [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_floatingips': [{'id': 'f7e7609b-a65e-451e-8db6-b6e1bcbaa2e9', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'floating_ip_address': '172.27.0.168', 'floating_network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'router_id': '4969b258-f606-4def-b666-09764c475749', 'port_id': 'ca396585-cdb5-4a50-a4c9-d5954043208e', 'fixed_ip_address': '172.20.0.5', 'status': 'ACTIVE', 'standard_attr_id': 569, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'mac_address': 'fa:16:3e:37:08:e4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-20T04:29:15Z', 'updated_at': '2021-05-20T04:29:18Z', 'revision_number': 1, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:50Z', 'revision_number': 5, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:34:49.871546 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8b78a7db-48e4-4bc9-b3e0-de6a96d10d62 None None] New status for floating IP e1254a73-238b-496d-bda5-e60da4e8c2d4: DOWN {{(pid=91646) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 20 04:34:50.383537 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cdb97a16-aff6-41ff-9cee-73a1cf4b9ea5 req-05b69fa9-5495-4550-8dc0-630fef296406 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:34:50.385180 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cdb97a16-aff6-41ff-9cee-73a1cf4b9ea5 req-05b69fa9-5495-4550-8dc0-630fef296406 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0822201 May 20 04:34:50.434417 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-cdb97a16-aff6-41ff-9cee-73a1cf4b9ea5 req-a85e9c08-a942-42dc-b0f5-9a1673ed24c0 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0431645 May 20 04:34:50.601624 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-672304f2-85ca-4a69-b43e-57ed5ba22dba req-b8132c75-75ac-4d5f-a8c2-c114e37f9f73 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:34:50.603062 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-672304f2-85ca-4a69-b43e-57ed5ba22dba req-b8132c75-75ac-4d5f-a8c2-c114e37f9f73 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.74 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 945 time: 0.0979404 May 20 04:34:50.744061 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-672304f2-85ca-4a69-b43e-57ed5ba22dba req-911e076d-8855-4939-a913-3830ead896e7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Deleting port fb2044c2-038a-473c-968f-29ae50d3c1a4 {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:34:50.744584 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-672304f2-85ca-4a69-b43e-57ed5ba22dba req-911e076d-8855-4939-a913-3830ead896e7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:50.888076 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [req-672304f2-85ca-4a69-b43e-57ed5ba22dba req-911e076d-8855-4939-a913-3830ead896e7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] For port fb2044c2-038a-473c-968f-29ae50d3c1a4, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=fb2044c2-038a-473c-968f-29ae50d3c1a4,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:34:50.888430 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-672304f2-85ca-4a69-b43e-57ed5ba22dba req-911e076d-8855-4939-a913-3830ead896e7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:51.020442 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [req-672304f2-85ca-4a69-b43e-57ed5ba22dba req-911e076d-8855-4939-a913-3830ead896e7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Calling delete_port for fb2044c2-038a-473c-968f-29ae50d3c1a4 owned by compute:nova {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:34:51.143018 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-672304f2-85ca-4a69-b43e-57ed5ba22dba req-911e076d-8855-4939-a913-3830ead896e7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:51.151332 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-56ed50e2-ae99-461d-b0bd-5a15bd728de6 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:51.347556 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-911e076d-8855-4939-a913-3830ead896e7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event deleted for resources: {'Port': ['ID=fb2044c2-038a-473c-968f-29ae50d3c1a4,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:51.350193 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-911e076d-8855-4939-a913-3830ead896e7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.199s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:51.390652 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [req-672304f2-85ca-4a69-b43e-57ed5ba22dba req-911e076d-8855-4939-a913-3830ead896e7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] No DVR routers for this DVR port fb2044c2-038a-473c-968f-29ae50d3c1a4 on host hv-cinder-78240 {{(pid=91643) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:34:51.392367 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [req-672304f2-85ca-4a69-b43e-57ed5ba22dba req-911e076d-8855-4939-a913-3830ead896e7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:51.396407 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '7cc291a5-7b2b-48ad-af24-f0fbbbf92685', 'name': 'network-vif-deleted', 'tag': 'fb2044c2-038a-473c-968f-29ae50d3c1a4'}] {{(pid=91643) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:34:51.411616 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-672304f2-85ca-4a69-b43e-57ed5ba22dba req-911e076d-8855-4939-a913-3830ead896e7 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.74 "DELETE /v2.0/ports/fb2044c2-038a-473c-968f-29ae50d3c1a4 HTTP/1.1" status: 204 len: 173 time: 0.8019409 May 20 04:34:51.461111 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '7cc291a5-7b2b-48ad-af24-f0fbbbf92685', 'name': 'network-vif-deleted', 'tag': 'fb2044c2-038a-473c-968f-29ae50d3c1a4', 'status': 'completed', 'code': 200} May 20 04:34:51.599090 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-748dc27a-8369-457a-a099-eb849d588d15 req-b0d5ef13-56ce-4d61-894a-0a8b921a962c tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Synchronizing usage tracker for tenant:0e3baa0ea24d42f3ab59e40845b350fd on resource:port {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:34:51.616803 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-748dc27a-8369-457a-a099-eb849d588d15 req-b0d5ef13-56ce-4d61-894a-0a8b921a962c tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Unset dirty status for tenant:0e3baa0ea24d42f3ab59e40845b350fd on resource:port {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:34:51.618106 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-748dc27a-8369-457a-a099-eb849d588d15 req-b0d5ef13-56ce-4d61-894a-0a8b921a962c tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=7cc291a5-7b2b-48ad-af24-f0fbbbf92685 HTTP/1.1" status: 200 len: 210 time: 0.0649867 May 20 04:34:53.584895 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device fb2044c2-038a-473c-968f-29ae50d3c1a4 no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:34:53.625772 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: fb2044c2-038a-473c-968f-29ae50d3c1a4 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:34:53.626111 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device fb2044c2-038a-473c-968f-29ae50d3c1a4 not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:34:55.148018 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 44820) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:55.501145 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-7d4057e8-bd63-4557-af65-2d83ca7eed3f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Deleting port c99984e7-2cf8-48d4-b093-fca7f6aef3ac {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:34:55.501471 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-7d4057e8-bd63-4557-af65-2d83ca7eed3f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:55.627543 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-7d4057e8-bd63-4557-af65-2d83ca7eed3f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:55.627875 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-7d4057e8-bd63-4557-af65-2d83ca7eed3f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:55.749291 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-7d4057e8-bd63-4557-af65-2d83ca7eed3f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Calling delete_port for c99984e7-2cf8-48d4-b093-fca7f6aef3ac owned by network:floatingip {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:34:55.879242 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-7d4057e8-bd63-4557-af65-2d83ca7eed3f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:55.883020 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-be0e0973-ebf8-48c1-b1fc-92cd412d56eb admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:55.930077 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7d4057e8-bd63-4557-af65-2d83ca7eed3f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event deleted for resources: {'Port': ['ID=c99984e7-2cf8-48d4-b093-fca7f6aef3ac,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:55.932726 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-7d4057e8-bd63-4557-af65-2d83ca7eed3f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:56.025006 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-7d4057e8-bd63-4557-af65-2d83ca7eed3f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for floatingip, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:56.025366 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.db.l3_db [None req-7d4057e8-bd63-4557-af65-2d83ca7eed3f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Floating IP f7e7609b-a65e-451e-8db6-b6e1bcbaa2e9 disassociated (deleted). External IP: 172.27.0.168, port: ca396585-cdb5-4a50-a4c9-d5954043208e. May 20 04:34:56.377173 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-7d4057e8-bd63-4557-af65-2d83ca7eed3f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Router 4969b258-f606-4def-b666-09764c475749 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91644) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:34:56.561491 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-7d4057e8-bd63-4557-af65-2d83ca7eed3f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:34:56.572961 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-41352a1d-d33e-46d0-9709-c6508de51c4b None None] Sync routers for ids ['4969b258-f606-4def-b666-09764c475749'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:34:56.584375 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-41352a1d-d33e-46d0-9709-c6508de51c4b None None] Routers {'4969b258-f606-4def-b666-09764c475749'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:34:56.585001 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-41352a1d-d33e-46d0-9709-c6508de51c4b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:34:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['4969b258-f606-4def-b666-09764c475749'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:34:56.587167 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-7d4057e8-bd63-4557-af65-2d83ca7eed3f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:56.768224 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-7d4057e8-bd63-4557-af65-2d83ca7eed3f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "DELETE /v2.0/floatingips/f7e7609b-a65e-451e-8db6-b6e1bcbaa2e9 HTTP/1.1" status: 204 len: 168 time: 1.6191242 May 20 04:34:56.769086 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'tag': 'ca396585-cdb5-4a50-a4c9-d5954043208e'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:34:56.843774 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'tag': 'ca396585-cdb5-4a50-a4c9-d5954043208e', 'status': 'completed', 'code': 200} May 20 04:34:56.872638 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-41352a1d-d33e-46d0-9709-c6508de51c4b None None] Set router 4969b258-f606-4def-b666-09764c475749 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:34:56.893195 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 44858) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:56.987118 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9662e876-af2c-4452-8ff1-0b0a05b22ccc req-0e287dc4-41bf-4300-aca5-a30c2486daa1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:34:56.988734 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9662e876-af2c-4452-8ff1-0b0a05b22ccc req-0e287dc4-41bf-4300-aca5-a30c2486daa1 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=0e3baa0ea24d42f3ab59e40845b350fd&device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 1098 time: 0.1260581 May 20 04:34:57.060019 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e7dfce89-0f6d-40a0-a27c-287ef3ca028e admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:57.103721 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-e7dfce89-0f6d-40a0-a27c-287ef3ca028e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:34:57.119040 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-e7dfce89-0f6d-40a0-a27c-287ef3ca028e admin admin] Usage tracker for resource:security_group_rule and tenant:9c24bc14b42f41bf9cfafa2c1cc1bc43 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:34:57.128422 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-e7dfce89-0f6d-40a0-a27c-287ef3ca028e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:34:57.141581 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-e7dfce89-0f6d-40a0-a27c-287ef3ca028e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:34:57.255789 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-41352a1d-d33e-46d0-9709-c6508de51c4b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:50Z', 'revision_number': 5, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:34:57.257036 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e7dfce89-0f6d-40a0-a27c-287ef3ca028e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:57.260442 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-41352a1d-d33e-46d0-9709-c6508de51c4b None None] Checking router: 4969b258-f606-4def-b666-09764c475749 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:34:57.263072 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e7dfce89-0f6d-40a0-a27c-287ef3ca028e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:57.269355 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-41352a1d-d33e-46d0-9709-c6508de51c4b None None] Routers {'4969b258-f606-4def-b666-09764c475749'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:34:57.269978 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-41352a1d-d33e-46d0-9709-c6508de51c4b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:34:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['4969b258-f606-4def-b666-09764c475749'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:34:57.296448 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e1a6a008-61fd-46da-a558-d1db426d91e7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:57.296691 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9662e876-af2c-4452-8ff1-0b0a05b22ccc req-c1a73ed5-6e4c-4d34-8098-a9eafc8f4106 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:34:57.298218 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9662e876-af2c-4452-8ff1-0b0a05b22ccc req-c1a73ed5-6e4c-4d34-8098-a9eafc8f4106 service neutron] 10.222.0.74 "GET /v2.0/networks?id=14c537db-01aa-4360-a348-47f94c75e03b HTTP/1.1" status: 200 len: 911 time: 0.3025553 May 20 04:34:57.322332 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-e7dfce89-0f6d-40a0-a27c-287ef3ca028e admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=9c24bc14b42f41bf9cfafa2c1cc1bc43&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4279964 May 20 04:34:57.325457 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 44872) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:57.337618 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e7dfce89-0f6d-40a0-a27c-287ef3ca028e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=89878f91-e547-4751-baea-23b94f2b33e3,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:57.343952 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e7dfce89-0f6d-40a0-a27c-287ef3ca028e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:57.384851 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-996c873b-b4d0-4646-9b83-1a0b6df0fc2c req-420707ec-e564-4752-919c-c22c43597429 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:34:57.387256 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-996c873b-b4d0-4646-9b83-1a0b6df0fc2c req-420707ec-e564-4752-919c-c22c43597429 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 945 time: 0.2078440 May 20 04:34:57.397865 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9662e876-af2c-4452-8ff1-0b0a05b22ccc req-2ec53f8d-415f-4a36-a0dc-ce483bb1bee4 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=ca396585-cdb5-4a50-a4c9-d5954043208e HTTP/1.1" status: 200 len: 217 time: 0.0898750 May 20 04:34:57.409988 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ae48dbcb-7f03-4f91-b0aa-d8854e7042f6 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:57.436583 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ae48dbcb-7f03-4f91-b0aa-d8854e7042f6 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:57.469163 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-41352a1d-d33e-46d0-9709-c6508de51c4b None None] Set router 4969b258-f606-4def-b666-09764c475749 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:34:57.493850 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ae48dbcb-7f03-4f91-b0aa-d8854e7042f6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:57.498661 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-996c873b-b4d0-4646-9b83-1a0b6df0fc2c req-88c0833e-1832-4485-82cf-d3602cde3c7d tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/security-groups?id=5a7943e2-185b-4112-9281-61ab71b1afeb&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1026978 May 20 04:34:57.499844 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e3916fb5-11b7-4a76-be10-cfa0908f1ff2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:57.502797 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9662e876-af2c-4452-8ff1-0b0a05b22ccc req-50290192-9205-4c66-bdd5-1e81ef1191d5 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:34:57.504246 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9662e876-af2c-4452-8ff1-0b0a05b22ccc req-50290192-9205-4c66-bdd5-1e81ef1191d5 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=cdea475c-2929-4ae4-9271-87cc2fb9a378 HTTP/1.1" status: 200 len: 849 time: 0.0920660 May 20 04:34:57.515644 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ae48dbcb-7f03-4f91-b0aa-d8854e7042f6 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=89878f91-e547-4751-baea-23b94f2b33e3,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:57.524388 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-ae48dbcb-7f03-4f91-b0aa-d8854e7042f6 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:57.526060 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-ae48dbcb-7f03-4f91-b0aa-d8854e7042f6 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/89878f91-e547-4751-baea-23b94f2b33e3 HTTP/1.1" status: 204 len: 168 time: 0.1997070 May 20 04:34:57.527320 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-ae48dbcb-7f03-4f91-b0aa-d8854e7042f6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:57.585435 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9662e876-af2c-4452-8ff1-0b0a05b22ccc req-3e59faa6-01b3-4932-a711-0296aa467890 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:34:57.587143 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9662e876-af2c-4452-8ff1-0b0a05b22ccc req-3e59faa6-01b3-4932-a711-0296aa467890 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=14c537db-01aa-4360-a348-47f94c75e03b&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0756998 May 20 04:34:57.723978 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9662e876-af2c-4452-8ff1-0b0a05b22ccc req-1808c62f-b387-4d73-8024-f415ecbf8d86 service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1286397 May 20 04:34:57.806044 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-41352a1d-d33e-46d0-9709-c6508de51c4b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:50Z', 'revision_number': 5, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:34:57.809664 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-41352a1d-d33e-46d0-9709-c6508de51c4b None None] The sync data for ids ['4969b258-f606-4def-b666-09764c475749'] in dv-cinder-782405-9-iscsi is: [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:24:38Z', 'revision_number': 4, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi', '_interfaces': [{'id': '8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7', 'name': '', 'network_id': '14c537db-01aa-4360-a348-47f94c75e03b', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'mac_address': 'fa:16:3e:8e:4f:00', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_interface', 'standard_attr_id': 308, 'fixed_ips': [{'subnet_id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:37Z', 'updated_at': '2021-05-20T04:24:50Z', 'revision_number': 5, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'subnets': [{'id': 'cdea475c-2929-4ae4-9271-87cc2fb9a378', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:34:57.874217 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9662e876-af2c-4452-8ff1-0b0a05b22ccc req-beb4b4ec-df29-4011-a314-c25ed4bb0bef service neutron] 10.222.0.74 "GET /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1398430 May 20 04:34:57.931036 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d18f0476-2069-4d54-b66a-f5bdb36efa9e req-9f2a089a-9317-42f7-85dc-a31cb7814127 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:34:57.931819 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-d18f0476-2069-4d54-b66a-f5bdb36efa9e req-9f2a089a-9317-42f7-85dc-a31cb7814127 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Synchronizing usage tracker for tenant:0e3baa0ea24d42f3ab59e40845b350fd on resource:port {{(pid=91644) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:34:57.948389 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-d18f0476-2069-4d54-b66a-f5bdb36efa9e req-9f2a089a-9317-42f7-85dc-a31cb7814127 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Unset dirty status for tenant:0e3baa0ea24d42f3ab59e40845b350fd on resource:port {{(pid=91644) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:34:57.949996 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-d18f0476-2069-4d54-b66a-f5bdb36efa9e req-9f2a089a-9317-42f7-85dc-a31cb7814127 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.74 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 945 time: 0.1115992 May 20 04:34:58.065626 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-63be76c1-a162-4e02-8ef0-a02cc26bbd31 None None] New status for floating IP f7e7609b-a65e-451e-8db6-b6e1bcbaa2e9: DOWN {{(pid=91646) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 20 04:34:58.106395 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [req-d18f0476-2069-4d54-b66a-f5bdb36efa9e req-6b0bd82f-486c-42f8-ab96-227bfb89573f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Deleting port ca396585-cdb5-4a50-a4c9-d5954043208e {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:34:58.106792 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-d18f0476-2069-4d54-b66a-f5bdb36efa9e req-6b0bd82f-486c-42f8-ab96-227bfb89573f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:58.244616 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-d18f0476-2069-4d54-b66a-f5bdb36efa9e req-6b0bd82f-486c-42f8-ab96-227bfb89573f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] For port ca396585-cdb5-4a50-a4c9-d5954043208e, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=ca396585-cdb5-4a50-a4c9-d5954043208e,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:34:58.244969 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-d18f0476-2069-4d54-b66a-f5bdb36efa9e req-6b0bd82f-486c-42f8-ab96-227bfb89573f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:58.259022 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 44896) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:58.363515 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.v2.base [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Request body: {'network': {'name': 'tempest-VolumesSnapshotTestJSON-1131429696-network', 'tenant_id': '1cc8a6f23e014d75bb434e4e526a41a2'}} {{(pid=91643) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:34:58.366286 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [req-d18f0476-2069-4d54-b66a-f5bdb36efa9e req-6b0bd82f-486c-42f8-ab96-227bfb89573f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Calling delete_port for ca396585-cdb5-4a50-a4c9-d5954043208e owned by compute:nova {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:34:58.368378 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:34:58.373420 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Usage tracker for resource:network and tenant:1cc8a6f23e014d75bb434e4e526a41a2 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:34:58.376713 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:34:58.383771 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:34:58.394697 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Made reservation on behalf of 1cc8a6f23e014d75bb434e4e526a41a2 for: {'network': 1} {{(pid=91643) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:34:58.397122 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-585852'] for network, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:58.409430 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:58.437361 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:34:58.437361 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Usage tracker for resource:security_group_rule and tenant:1cc8a6f23e014d75bb434e4e526a41a2 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:34:58.437361 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:34:58.441107 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:34:58.486166 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-d18f0476-2069-4d54-b66a-f5bdb36efa9e req-6b0bd82f-486c-42f8-ab96-227bfb89573f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:58.492121 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-cb99be19-b420-4c3a-94ca-8aa7437f6764 tempest-VolumesActionsTest-1036064425 tempest-VolumesActionsTest-1036064425-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:58.541321 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6b0bd82f-486c-42f8-ab96-227bfb89573f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event deleted for resources: {'Port': ['ID=ca396585-cdb5-4a50-a4c9-d5954043208e,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:58.544555 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-6b0bd82f-486c-42f8-ab96-227bfb89573f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:58.577738 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:58.586075 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:58.590973 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-59dd78bb-be3d-40e5-b282-f518d2039271 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:58.618964 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] vlan segment allocate from pool success with {'physical_network': 'physnet1', 'vlan_id': 830} {{(pid=91643) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 20 04:34:58.627173 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-d18f0476-2069-4d54-b66a-f5bdb36efa9e req-6b0bd82f-486c-42f8-ab96-227bfb89573f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] No DVR routers for this DVR port ca396585-cdb5-4a50-a4c9-d5954043208e on host hv-cinder-78240 {{(pid=91644) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:34:58.630540 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-d18f0476-2069-4d54-b66a-f5bdb36efa9e req-6b0bd82f-486c-42f8-ab96-227bfb89573f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:58.632969 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=6f0d273f-0b64-4321-8661-9765ae71fbcb,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:58.645544 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:58.646753 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-d18f0476-2069-4d54-b66a-f5bdb36efa9e req-6b0bd82f-486c-42f8-ab96-227bfb89573f tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.74 "DELETE /v2.0/ports/ca396585-cdb5-4a50-a4c9-d5954043208e HTTP/1.1" status: 204 len: 173 time: 0.6907589 May 20 04:34:58.686974 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:58.687664 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.db.segments_db [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Added segment 76847180-0644-4d57-b40f-7f63a056f8ec of type vlan for network bb372f0c-268d-4c71-bb09-cab162d5731a May 20 04:34:58.703863 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-be37bbff-77f5-4039-84a8-dd47b686a0aa req-7992f986-3e79-42bb-9feb-69706ceaabb2 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "GET /v2.0/ports?device_id=4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5 HTTP/1.1" status: 200 len: 210 time: 0.0571067 May 20 04:34:58.728319 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8760688516720'] for network, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:58.762071 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915'] for network, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:58.763348 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:34:58.763925 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:58.769025 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-f79aaf39-aefc-4640-aefc-c58e4dedcd6f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:58.844824 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'name': 'network-vif-deleted', 'tag': 'ca396585-cdb5-4a50-a4c9-d5954043208e'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:34:58.868840 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] 10.222.0.39 "POST /v2.0/networks HTTP/1.1" status: 201 len: 887 time: 0.6086085 May 20 04:34:58.869812 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Pushing event updated for resources: {'Network': ['ID=bb372f0c-268d-4c71-bb09-cab162d5731a,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:58.871927 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 44916) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:58.872722 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-94e6534d-0ca5-4922-96f4-84b42d104185 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:58.878449 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [None req-3afc42ed-9e3c-4298-b2f2-20cd1dcc251a admin admin] Request body: {'subnet': {'network_id': 'bb372f0c-268d-4c71-bb09-cab162d5731a', 'cidr': '172.20.0.0/28', 'name': 'tempest-VolumesSnapshotTestJSON-1131429696-subnet', 'tenant_id': '1cc8a6f23e014d75bb434e4e526a41a2', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:34:58.887058 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Network bb372f0c-268d-4c71-bb09-cab162d5731a details requested by agent ovs-agent-dv-cinder-782405-9-iscsi with host dv-cinder-782405-9-iscsi {{(pid=91645) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 20 04:34:58.891984 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-3afc42ed-9e3c-4298-b2f2-20cd1dcc251a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:34:58.902381 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-3afc42ed-9e3c-4298-b2f2-20cd1dcc251a admin admin] Usage tracker for resource:subnet and tenant:1cc8a6f23e014d75bb434e4e526a41a2 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:34:58.910873 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-3afc42ed-9e3c-4298-b2f2-20cd1dcc251a admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:34:58.925272 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '4b0dc068-c4b0-4a88-9e48-cda37cd8f2c5', 'name': 'network-vif-deleted', 'tag': 'ca396585-cdb5-4a50-a4c9-d5954043208e', 'status': 'completed', 'code': 200} May 20 04:34:58.926311 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-3afc42ed-9e3c-4298-b2f2-20cd1dcc251a admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:34:58.951984 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-3afc42ed-9e3c-4298-b2f2-20cd1dcc251a admin admin] Made reservation on behalf of 1cc8a6f23e014d75bb434e4e526a41a2 for: {'subnet': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:34:58.954457 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-3afc42ed-9e3c-4298-b2f2-20cd1dcc251a admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-1810190'] for subnet, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:59.225907 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 44926) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:59.344881 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3ab29511-b45b-410c-a857-9a0b292816be admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:59.360336 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-3ab29511-b45b-410c-a857-9a0b292816be admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:34:59.364277 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-3ab29511-b45b-410c-a857-9a0b292816be admin admin] Usage tracker for resource:security_group_rule and tenant:bad89fc8e0124c93b2be511252105e62 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:34:59.367198 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-3ab29511-b45b-410c-a857-9a0b292816be admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:34:59.373612 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-3ab29511-b45b-410c-a857-9a0b292816be admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:34:59.434338 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-3afc42ed-9e3c-4298-b2f2-20cd1dcc251a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-1988978', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-1810236'] for subnet, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:59.439123 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:59.468085 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3ab29511-b45b-410c-a857-9a0b292816be admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:59.474260 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-3ab29511-b45b-410c-a857-9a0b292816be admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:59.484120 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-44cd32a5-bf0c-4840-9608-5c50289a01f7 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:59.514012 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3afc42ed-9e3c-4298-b2f2-20cd1dcc251a admin admin] Pushing event updated for resources: {'Subnet': ['ID=953afe7b-ba49-4dd0-b9aa-085c509e22c4,revision_number=0']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:59.517717 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-3afc42ed-9e3c-4298-b2f2-20cd1dcc251a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:59.520166 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3afc42ed-9e3c-4298-b2f2-20cd1dcc251a admin admin] Checking segment: {'id': '76847180-0644-4d57-b40f-7f63a056f8ec', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 830, 'network_id': 'bb372f0c-268d-4c71-bb09-cab162d5731a'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:34:59.520486 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-3afc42ed-9e3c-4298-b2f2-20cd1dcc251a admin admin] Network hostable DHCP agents. Network: bb372f0c-268d-4c71-bb09-cab162d5731a, hostable agents: ['fffae25a-f450-4182-8f9e-ac29d143f6c9'], hosted agents: [] {{(pid=91644) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 20 04:34:59.526829 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-3ab29511-b45b-410c-a857-9a0b292816be admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=bad89fc8e0124c93b2be511252105e62&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2982018 May 20 04:34:59.526829 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3ab29511-b45b-410c-a857-9a0b292816be admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=07b95e5a-f2f0-418c-a713-004c3aaf2877,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:59.528400 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 44932) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:59.529493 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-3ab29511-b45b-410c-a857-9a0b292816be admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:59.564742 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-3afc42ed-9e3c-4298-b2f2-20cd1dcc251a admin admin] Network bb372f0c-268d-4c71-bb09-cab162d5731a is scheduled to be hosted by DHCP agent fffae25a-f450-4182-8f9e-ac29d143f6c9 with binding_index 1 {{(pid=91644) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 20 04:34:59.585758 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [None req-3afc42ed-9e3c-4298-b2f2-20cd1dcc251a admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-782405-9-iscsi, bb372f0c-268d-4c71-bb09-cab162d5731a) {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:34:59.595141 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-36902bce-dc8f-4f1a-a2c1-7e77f9cd1db8 None None] Network bb372f0c-268d-4c71-bb09-cab162d5731a requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:34:59.605877 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-3afc42ed-9e3c-4298-b2f2-20cd1dcc251a admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:34:59.606459 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-3afc42ed-9e3c-4298-b2f2-20cd1dcc251a admin admin] Notify callbacks [] for subnet, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:59.609782 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-af082ed0-eed8-47aa-9dde-43f79b63c32e admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:59.648561 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-af082ed0-eed8-47aa-9dde-43f79b63c32e admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:59.656234 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-3afc42ed-9e3c-4298-b2f2-20cd1dcc251a admin admin] 10.222.0.39 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 851 time: 0.7828195 May 20 04:34:59.712799 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-af082ed0-eed8-47aa-9dde-43f79b63c32e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:59.717977 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-6844da9e-de6d-46da-bb84-73a9ad44c140 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:34:59.731665 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-af082ed0-eed8-47aa-9dde-43f79b63c32e admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=07b95e5a-f2f0-418c-a713-004c3aaf2877,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:34:59.734412 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-af082ed0-eed8-47aa-9dde-43f79b63c32e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:34:59.736058 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-af082ed0-eed8-47aa-9dde-43f79b63c32e admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:34:59.737151 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-af082ed0-eed8-47aa-9dde-43f79b63c32e admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/07b95e5a-f2f0-418c-a713-004c3aaf2877 HTTP/1.1" status: 204 len: 168 time: 0.2078087 May 20 04:34:59.822799 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 44942) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:34:59.858469 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-3e8936ce-3a94-481d-8ad4-e86ab1333560 None None] Network bb372f0c-268d-4c71-bb09-cab162d5731a requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:35:00.022857 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:35:00.106165 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 44952) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:00.138349 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-133dcbbf-65a6-406a-9726-dd3c0e5e54df admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:00.153089 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-133dcbbf-65a6-406a-9726-dd3c0e5e54df admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:35:00.155305 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-d48c3948-7a6b-49c3-8a97-994eb8227138 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Deleting port a2787f07-3708-4444-9177-fd660f18fb66 {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:35:00.155760 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d48c3948-7a6b-49c3-8a97-994eb8227138 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:00.156527 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-133dcbbf-65a6-406a-9726-dd3c0e5e54df admin admin] Usage tracker for resource:security_group_rule and tenant:afc81cde9c17444983657ec3e1b8fc5f is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:35:00.159396 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-133dcbbf-65a6-406a-9726-dd3c0e5e54df admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:35:00.165471 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-133dcbbf-65a6-406a-9726-dd3c0e5e54df admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:35:00.253042 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-133dcbbf-65a6-406a-9726-dd3c0e5e54df admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:00.257717 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-133dcbbf-65a6-406a-9726-dd3c0e5e54df admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:00.266315 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-a48ef099-fc5b-4820-83ee-7349dfd6c098 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:00.275538 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d48c3948-7a6b-49c3-8a97-994eb8227138 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:00.275879 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d48c3948-7a6b-49c3-8a97-994eb8227138 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:00.305353 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-133dcbbf-65a6-406a-9726-dd3c0e5e54df admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=afc81cde9c17444983657ec3e1b8fc5f&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1976259 May 20 04:35:00.307832 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 44956) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:00.310461 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-133dcbbf-65a6-406a-9726-dd3c0e5e54df admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e1e27eae-c71f-412f-acfa-89e0c346def9,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:00.322862 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-133dcbbf-65a6-406a-9726-dd3c0e5e54df admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.057s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:00.378820 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c7c362d7-f0f2-4f43-978a-7a2db28ea702 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:00.401739 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-d48c3948-7a6b-49c3-8a97-994eb8227138 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Calling delete_port for a2787f07-3708-4444-9177-fd660f18fb66 owned by network:floatingip {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:35:00.406130 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c7c362d7-f0f2-4f43-978a-7a2db28ea702 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:00.463472 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c7c362d7-f0f2-4f43-978a-7a2db28ea702 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:00.474395 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-133dcbbf-65a6-406a-9726-dd3c0e5e54df admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:00.490345 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c7c362d7-f0f2-4f43-978a-7a2db28ea702 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e1e27eae-c71f-412f-acfa-89e0c346def9,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:00.492693 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-c7c362d7-f0f2-4f43-978a-7a2db28ea702 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:00.493621 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-c7c362d7-f0f2-4f43-978a-7a2db28ea702 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:00.494802 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-c7c362d7-f0f2-4f43-978a-7a2db28ea702 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/e1e27eae-c71f-412f-acfa-89e0c346def9 HTTP/1.1" status: 204 len: 168 time: 0.1860037 May 20 04:35:00.547954 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d48c3948-7a6b-49c3-8a97-994eb8227138 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:00.561305 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-c6ffb807-1557-4c96-a83a-3e0bac6e5c22 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:00.675834 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d48c3948-7a6b-49c3-8a97-994eb8227138 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event deleted for resources: {'Port': ['ID=a2787f07-3708-4444-9177-fd660f18fb66,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:00.678735 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d48c3948-7a6b-49c3-8a97-994eb8227138 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.118s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:00.707827 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d48c3948-7a6b-49c3-8a97-994eb8227138 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for floatingip, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:00.722772 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-d48c3948-7a6b-49c3-8a97-994eb8227138 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for floatingip, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:00.724261 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-d48c3948-7a6b-49c3-8a97-994eb8227138 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "DELETE /v2.0/floatingips/d2b40541-a315-4701-99bd-be17179c9b8b HTTP/1.1" status: 204 len: 168 time: 0.8999832 May 20 04:35:00.859359 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 44972) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:00.878658 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6efecb32-7159-4026-bd2d-a87baa0afc07 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:00.893595 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-6efecb32-7159-4026-bd2d-a87baa0afc07 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:35:00.897665 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-6efecb32-7159-4026-bd2d-a87baa0afc07 admin admin] Usage tracker for resource:security_group_rule and tenant:f437a6409cbf4a15b5925d52fc1b2aa4 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:35:00.900616 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-6efecb32-7159-4026-bd2d-a87baa0afc07 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:35:00.906599 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-6efecb32-7159-4026-bd2d-a87baa0afc07 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:35:00.993095 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6efecb32-7159-4026-bd2d-a87baa0afc07 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:00.997341 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6efecb32-7159-4026-bd2d-a87baa0afc07 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:01.005719 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-727d0d19-c9ea-4a3d-973d-fd34da44b6c8 tempest-AttachVolumeShelveTestJSON-1777644332 tempest-AttachVolumeShelveTestJSON-1777644332-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:01.047467 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-6efecb32-7159-4026-bd2d-a87baa0afc07 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=f437a6409cbf4a15b5925d52fc1b2aa4&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1868038 May 20 04:35:01.049090 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6efecb32-7159-4026-bd2d-a87baa0afc07 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f935ee36-92ad-4c2e-ad92-b9e61c944351,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:01.050218 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 44980) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:01.052062 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-6efecb32-7159-4026-bd2d-a87baa0afc07 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:01.113158 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-3b45077f-5266-44df-93ef-32da6627c10d admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:01.138389 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-3b45077f-5266-44df-93ef-32da6627c10d admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:01.189868 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-3b45077f-5266-44df-93ef-32da6627c10d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:01.193437 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e7dfce89-0f6d-40a0-a27c-287ef3ca028e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:01.204378 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3b45077f-5266-44df-93ef-32da6627c10d admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f935ee36-92ad-4c2e-ad92-b9e61c944351,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:01.207963 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-3b45077f-5266-44df-93ef-32da6627c10d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:01.211267 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-3b45077f-5266-44df-93ef-32da6627c10d admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:01.212353 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-3b45077f-5266-44df-93ef-32da6627c10d admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/f935ee36-92ad-4c2e-ad92-b9e61c944351 HTTP/1.1" status: 204 len: 168 time: 0.1611326 May 20 04:35:01.613566 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device ca396585-cdb5-4a50-a4c9-d5954043208e no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:35:01.654224 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: ca396585-cdb5-4a50-a4c9-d5954043208e {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:35:01.654575 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device ca396585-cdb5-4a50-a4c9-d5954043208e not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:35:02.103305 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 45022) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:02.112103 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9852a495-86f9-451a-894e-55ff269cda1f req-78b0c14a-d33f-4463-9742-e9aa98b9d352 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:02.112798 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-9852a495-86f9-451a-894e-55ff269cda1f req-78b0c14a-d33f-4463-9742-e9aa98b9d352 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Synchronizing usage tracker for tenant:1cc8a6f23e014d75bb434e4e526a41a2 on resource:network {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:35:02.133805 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-9852a495-86f9-451a-894e-55ff269cda1f req-78b0c14a-d33f-4463-9742-e9aa98b9d352 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Unset dirty status for tenant:1cc8a6f23e014d75bb434e4e526a41a2 on resource:network {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:35:02.135190 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9852a495-86f9-451a-894e-55ff269cda1f req-78b0c14a-d33f-4463-9742-e9aa98b9d352 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/networks?id=bb372f0c-268d-4c71-bb09-cab162d5731a HTTP/1.1" status: 200 len: 814 time: 0.2712467 May 20 04:35:02.151813 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9852a495-86f9-451a-894e-55ff269cda1f req-234e4311-d21d-4a4e-801b-d8576f43e012 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/quotas/1cc8a6f23e014d75bb434e4e526a41a2 HTTP/1.1" status: 200 len: 375 time: 0.0103188 May 20 04:35:02.185673 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-7e704f33-5a34-401c-b8cc-996cb977e7e3 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:02.200385 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9852a495-86f9-451a-894e-55ff269cda1f req-d159e297-a3a6-40f1-9403-59abe54c6e78 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=1cc8a6f23e014d75bb434e4e526a41a2&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0427575 May 20 04:35:02.215560 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-7e704f33-5a34-401c-b8cc-996cb977e7e3 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:02.294364 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-7e704f33-5a34-401c-b8cc-996cb977e7e3 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:02.299712 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-c80039e3-7ece-47f2-94b8-a49167903484 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:02.315714 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7e704f33-5a34-401c-b8cc-996cb977e7e3 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=5a7943e2-185b-4112-9281-61ab71b1afeb,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:02.319999 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [req-9852a495-86f9-451a-894e-55ff269cda1f req-c0516385-4d13-45b5-94d8-daa251ee69ac service neutron] http://10.222.0.39:9696/v2.0/extensions returned with HTTP 200 {{(pid=91643) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 20 04:35:02.321708 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-7e704f33-5a34-401c-b8cc-996cb977e7e3 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.021s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:02.327398 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9852a495-86f9-451a-894e-55ff269cda1f req-c0516385-4d13-45b5-94d8-daa251ee69ac service neutron] 10.222.0.39 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.1204402 May 20 04:35:02.331337 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-7e704f33-5a34-401c-b8cc-996cb977e7e3 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:02.332613 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-7e704f33-5a34-401c-b8cc-996cb977e7e3 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] 10.222.0.39 "DELETE /v2.0/security-groups/5a7943e2-185b-4112-9281-61ab71b1afeb HTTP/1.1" status: 204 len: 168 time: 0.2281773 May 20 04:35:02.399073 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 45038) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:02.470421 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9852a495-86f9-451a-894e-55ff269cda1f req-ce635bc9-32b8-4518-b6a9-2d56806bc93f service neutron] 10.222.0.39 "GET /v2.0/networks/bb372f0c-268d-4c71-bb09-cab162d5731a?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1421990 May 20 04:35:02.610649 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-9852a495-86f9-451a-894e-55ff269cda1f req-4bf1abaa-e3d7-4824-947a-d10b3bd9962f service neutron] 10.222.0.39 "GET /v2.0/networks/bb372f0c-268d-4c71-bb09-cab162d5731a?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1326902 May 20 04:35:03.046651 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-3c3bda92-1df2-4311-b8b9-b8e16e291e88 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-8003145'] for router_interface, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:03.403165 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 45070) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:03.553437 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8be579cd-ac64-4610-8118-6fcd7ae69f12 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:03.576981 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-8be579cd-ac64-4610-8118-6fcd7ae69f12 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:35:03.582118 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-8be579cd-ac64-4610-8118-6fcd7ae69f12 admin admin] Usage tracker for resource:security_group_rule and tenant:73ee3573e8004d61b31543e793127f52 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:35:03.585684 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-8be579cd-ac64-4610-8118-6fcd7ae69f12 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:35:03.593388 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-8be579cd-ac64-4610-8118-6fcd7ae69f12 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:35:03.620618 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-3c3bda92-1df2-4311-b8b9-b8e16e291e88 admin admin] Deleting port 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:35:03.620911 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-3c3bda92-1df2-4311-b8b9-b8e16e291e88 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:03.781297 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [None req-3c3bda92-1df2-4311-b8b9-b8e16e291e88 admin admin] For port 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:35:03.781645 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-3c3bda92-1df2-4311-b8b9-b8e16e291e88 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:03.811869 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-82a704a4-a81b-4aae-a643-6c400b51fc99 req-5cd91b4a-5004-4bb1-b9b9-2a32056baa11 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=06b8328b-158c-4b35-9bc3-95b9eab9515c HTTP/1.1" status: 200 len: 210 time: 0.0440321 May 20 04:35:03.908140 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8be579cd-ac64-4610-8118-6fcd7ae69f12 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:03.916521 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-3c3bda92-1df2-4311-b8b9-b8e16e291e88 admin admin] Calling delete_port for 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 owned by network:router_interface {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:35:03.920855 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-8be579cd-ac64-4610-8118-6fcd7ae69f12 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:03.928573 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-0402b0dd-e7c5-4352-9c9c-c8d843de3436 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:03.980937 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8be579cd-ac64-4610-8118-6fcd7ae69f12 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=a36d4582-85bd-4c56-be4c-3255a0a9f1cf,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:03.987608 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-8be579cd-ac64-4610-8118-6fcd7ae69f12 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=73ee3573e8004d61b31543e793127f52&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5809834 May 20 04:35:03.987608 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-8be579cd-ac64-4610-8118-6fcd7ae69f12 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.059s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:03.989036 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 45086) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:04.071206 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1bb089de-3b9e-4c25-8033-943ddb88e2c0 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:04.108256 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1bb089de-3b9e-4c25-8033-943ddb88e2c0 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:04.216731 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1bb089de-3b9e-4c25-8033-943ddb88e2c0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:04.222494 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-ae48dbcb-7f03-4f91-b0aa-d8854e7042f6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:04.248585 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1bb089de-3b9e-4c25-8033-943ddb88e2c0 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=a36d4582-85bd-4c56-be4c-3255a0a9f1cf,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:04.251728 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-1bb089de-3b9e-4c25-8033-943ddb88e2c0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.029s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:04.260745 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-3c3bda92-1df2-4311-b8b9-b8e16e291e88 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:04.266550 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:04.307249 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-1bb089de-3b9e-4c25-8033-943ddb88e2c0 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:04.308390 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-1bb089de-3b9e-4c25-8033-943ddb88e2c0 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/a36d4582-85bd-4c56-be4c-3255a0a9f1cf HTTP/1.1" status: 204 len: 168 time: 0.3184404 May 20 04:35:04.345472 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3c3bda92-1df2-4311-b8b9-b8e16e291e88 admin admin] Pushing event deleted for resources: {'Port': ['ID=8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:04.348857 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-3c3bda92-1df2-4311-b8b9-b8e16e291e88 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.082s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:04.607035 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-3c3bda92-1df2-4311-b8b9-b8e16e291e88 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-7924922', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-8054745'] for router_interface, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:04.639583 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 45100) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:04.658188 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-33b6f105-9d0e-4f51-b83d-f1a71af49332 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:04.677244 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-33b6f105-9d0e-4f51-b83d-f1a71af49332 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:35:04.681141 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-33b6f105-9d0e-4f51-b83d-f1a71af49332 admin admin] Usage tracker for resource:security_group_rule and tenant:399d27c700ff4053a1f74a8a75043d87 is out of sync, need to count used quota {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:35:04.684088 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [None req-33b6f105-9d0e-4f51-b83d-f1a71af49332 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91643) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:35:04.689774 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.quota.driver [None req-33b6f105-9d0e-4f51-b83d-f1a71af49332 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91643) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:35:04.774581 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-33b6f105-9d0e-4f51-b83d-f1a71af49332 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:04.780573 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-33b6f105-9d0e-4f51-b83d-f1a71af49332 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:04.788102 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-c7c362d7-f0f2-4f43-978a-7a2db28ea702 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:04.829549 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-33b6f105-9d0e-4f51-b83d-f1a71af49332 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=5ad0b791-a002-4d00-accf-11a2f18fed30,revision_number=1']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:04.834750 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-33b6f105-9d0e-4f51-b83d-f1a71af49332 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=399d27c700ff4053a1f74a8a75043d87&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1940808 May 20 04:35:04.836095 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-33b6f105-9d0e-4f51-b83d-f1a71af49332 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:04.837267 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 45106) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:04.898875 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-1d434756-ac7d-4cc9-8c26-db7df28b07e9 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:04.924459 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-1d434756-ac7d-4cc9-8c26-db7df28b07e9 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:05.008764 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-966e1adb-93f6-4ed1-946e-898f4a53242c req-9fd9de40-fd19-4a47-9f98-ae95ee36fff1 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=06b8328b-158c-4b35-9bc3-95b9eab9515c HTTP/1.1" status: 200 len: 210 time: 0.0553963 May 20 04:35:05.028361 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-1d434756-ac7d-4cc9-8c26-db7df28b07e9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:05.037521 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-a3ec278d-fab6-439e-9230-44b916f3bb21 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:05.051270 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1d434756-ac7d-4cc9-8c26-db7df28b07e9 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=5ad0b791-a002-4d00-accf-11a2f18fed30,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:05.060438 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-1d434756-ac7d-4cc9-8c26-db7df28b07e9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:05.062034 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-1d434756-ac7d-4cc9-8c26-db7df28b07e9 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:05.065803 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-1d434756-ac7d-4cc9-8c26-db7df28b07e9 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/5ad0b791-a002-4d00-accf-11a2f18fed30 HTTP/1.1" status: 204 len: 168 time: 0.2274325 May 20 04:35:05.248363 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9852a495-86f9-451a-894e-55ff269cda1f req-7d088f35-6388-42e0-9c23-5e8695d5c3cf tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:05.249899 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-9852a495-86f9-451a-894e-55ff269cda1f req-7d088f35-6388-42e0-9c23-5e8695d5c3cf tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.74 "GET /v2.0/networks?id=bb372f0c-268d-4c71-bb09-cab162d5731a HTTP/1.1" status: 200 len: 814 time: 0.2648518 May 20 04:35:05.257576 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-3c3bda92-1df2-4311-b8b9-b8e16e291e88 admin admin] Router 4969b258-f606-4def-b666-09764c475749 has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91644) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:35:05.264058 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-9852a495-86f9-451a-894e-55ff269cda1f req-bf01a353-e42e-4716-8d2e-032e245cb761 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Request body: {'port': {'device_id': '06b8328b-158c-4b35-9bc3-95b9eab9515c', 'network_id': 'bb372f0c-268d-4c71-bb09-cab162d5731a', 'admin_state_up': True, 'tenant_id': '1cc8a6f23e014d75bb434e4e526a41a2'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:35:05.544257 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-9852a495-86f9-451a-894e-55ff269cda1f req-bf01a353-e42e-4716-8d2e-032e245cb761 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:35:05.550991 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-9852a495-86f9-451a-894e-55ff269cda1f req-bf01a353-e42e-4716-8d2e-032e245cb761 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Usage tracker for resource:port and tenant:1cc8a6f23e014d75bb434e4e526a41a2 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:35:05.556165 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-9852a495-86f9-451a-894e-55ff269cda1f req-bf01a353-e42e-4716-8d2e-032e245cb761 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Quota usage for port was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:35:05.564621 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-9852a495-86f9-451a-894e-55ff269cda1f req-bf01a353-e42e-4716-8d2e-032e245cb761 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:35:05.567011 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-3c3bda92-1df2-4311-b8b9-b8e16e291e88 admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91644) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:35:05.572405 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-3c3bda92-1df2-4311-b8b9-b8e16e291e88 admin admin] 10.222.0.39 "PUT /v2.0/routers/4969b258-f606-4def-b666-09764c475749/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 3.1722543 May 20 04:35:05.575739 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 45126) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:05.577037 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-918196c6-be2e-4b92-a9be-e2ab9010a601 None None] Sync routers for ids ['4969b258-f606-4def-b666-09764c475749'] in dv-cinder-782405-9-iscsi {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:35:05.581029 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-9852a495-86f9-451a-894e-55ff269cda1f req-bf01a353-e42e-4716-8d2e-032e245cb761 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Made reservation on behalf of 1cc8a6f23e014d75bb434e4e526a41a2 for: {'port': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:35:05.585541 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-9852a495-86f9-451a-894e-55ff269cda1f req-bf01a353-e42e-4716-8d2e-032e245cb761 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:05.587959 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-918196c6-be2e-4b92-a9be-e2ab9010a601 None None] Routers {'4969b258-f606-4def-b666-09764c475749'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:35:05.588789 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-918196c6-be2e-4b92-a9be-e2ab9010a601 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:34:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['4969b258-f606-4def-b666-09764c475749'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:35:05.771770 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-de478202-6dfd-413c-815c-dee99190ad6e admin admin] Notify callbacks [] for router, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:05.775473 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [req-9852a495-86f9-451a-894e-55ff269cda1f req-bf01a353-e42e-4716-8d2e-032e245cb761 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Allocated IP 172.20.0.9 (bb372f0c-268d-4c71-bb09-cab162d5731a/953afe7b-ba49-4dd0-b9aa-085c509e22c4/7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:35:05.796234 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-918196c6-be2e-4b92-a9be-e2ab9010a601 None None] Set router 4969b258-f606-4def-b666-09764c475749 gateway port host: dv-cinder-782405-9-iscsi {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:35:05.956140 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-de478202-6dfd-413c-815c-dee99190ad6e admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:05.958041 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:06.032013 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-9852a495-86f9-451a-894e-55ff269cda1f req-bf01a353-e42e-4716-8d2e-032e245cb761 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:06.040182 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-de478202-6dfd-413c-815c-dee99190ad6e admin admin] Deleting port 8f9c794c-3d7c-41d8-8927-b7ccdfe087ae {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:35:06.040530 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-de478202-6dfd-413c-815c-dee99190ad6e admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:06.072785 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-9852a495-86f9-451a-894e-55ff269cda1f req-bf01a353-e42e-4716-8d2e-032e245cb761 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:06.080090 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-dbefaa97-177b-4108-adb3-4c8e1be78615 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:06.155979 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-9852a495-86f9-451a-894e-55ff269cda1f req-bf01a353-e42e-4716-8d2e-032e245cb761 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Network bb372f0c-268d-4c71-bb09-cab162d5731a is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:35:06.159898 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [req-9852a495-86f9-451a-894e-55ff269cda1f req-bf01a353-e42e-4716-8d2e-032e245cb761 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:35:06.176436 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bf01a353-e42e-4716-8d2e-032e245cb761 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Pushing event updated for resources: {'Port': ['ID=7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:06.177348 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-918196c6-be2e-4b92-a9be-e2ab9010a601 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210'}], 'enable_snat': True}, 'gw_port_id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:35:04Z', 'revision_number': 5, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port': {'id': '8f9c794c-3d7c-41d8-8927-b7ccdfe087ae', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:8a:eb:12', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '4969b258-f606-4def-b666-09764c475749', 'device_owner': 'network:router_gateway', 'standard_attr_id': 306, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.210', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:33Z', 'updated_at': '2021-05-20T04:24:44Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:35:06.188080 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-9852a495-86f9-451a-894e-55ff269cda1f req-bf01a353-e42e-4716-8d2e-032e245cb761 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:35:06.189263 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-918196c6-be2e-4b92-a9be-e2ab9010a601 None None] Checking router: 4969b258-f606-4def-b666-09764c475749 for host: dv-cinder-782405-9-iscsi {{(pid=91645) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:35:06.205991 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-918196c6-be2e-4b92-a9be-e2ab9010a601 None None] Routers {'4969b258-f606-4def-b666-09764c475749'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91645) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:35:06.206937 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-918196c6-be2e-4b92-a9be-e2ab9010a601 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:34:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['4969b258-f606-4def-b666-09764c475749'], 'active': True} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:35:06.222796 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [None req-de478202-6dfd-413c-815c-dee99190ad6e admin admin] For port 8f9c794c-3d7c-41d8-8927-b7ccdfe087ae, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=8f9c794c-3d7c-41d8-8927-b7ccdfe087ae,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:35:06.223179 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-de478202-6dfd-413c-815c-dee99190ad6e admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:06.256648 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9852a495-86f9-451a-894e-55ff269cda1f req-bf01a353-e42e-4716-8d2e-032e245cb761 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:06.257306 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-9852a495-86f9-451a-894e-55ff269cda1f req-bf01a353-e42e-4716-8d2e-032e245cb761 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:06.270031 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-bf01a353-e42e-4716-8d2e-032e245cb761 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.190s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:06.292288 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-9852a495-86f9-451a-894e-55ff269cda1f req-bf01a353-e42e-4716-8d2e-032e245cb761 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.0335131 May 20 04:35:06.307045 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Request body: {'port': {'device_id': '06b8328b-158c-4b35-9bc3-95b9eab9515c', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:35:06.387864 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvr_db [None req-918196c6-be2e-4b92-a9be-e2ab9010a601 None None] Set router 4969b258-f606-4def-b666-09764c475749 gateway port host: None {{(pid=91645) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:35:06.446674 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:06.468301 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-de478202-6dfd-413c-815c-dee99190ad6e admin admin] Calling delete_port for 8f9c794c-3d7c-41d8-8927-b7ccdfe087ae owned by network:router_gateway {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:35:06.517849 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_hamode_db [None req-918196c6-be2e-4b92-a9be-e2ab9010a601 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:35:05Z', 'revision_number': 6, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port_host': None}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91645) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:35:06.522260 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-918196c6-be2e-4b92-a9be-e2ab9010a601 None None] The sync data for ids ['4969b258-f606-4def-b666-09764c475749'] in dv-cinder-782405-9-iscsi is: [{'id': '4969b258-f606-4def-b666-09764c475749', 'name': 'tempest-TestVolumeBootPattern-59002837-router', 'tenant_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:32Z', 'updated_at': '2021-05-20T04:35:05Z', 'revision_number': 6, 'project_id': '0e3baa0ea24d42f3ab59e40845b350fd', 'gw_port_host': None}] {{(pid=91645) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:35:06.587168 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b12cbc34-7efc-435f-b135-8f8fbde0a55c req-88c7b708-d3f5-4e7a-8641-1d6e6516351a tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:06.588678 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b12cbc34-7efc-435f-b135-8f8fbde0a55c req-88c7b708-d3f5-4e7a-8641-1d6e6516351a tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=06b8328b-158c-4b35-9bc3-95b9eab9515c HTTP/1.1" status: 200 len: 931 time: 0.2951992 May 20 04:35:06.606570 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 45150) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:06.649074 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-b12cbc34-7efc-435f-b135-8f8fbde0a55c req-d5a041aa-885e-46ff-b819-e699f4fe4c4b tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Synchronizing usage tracker for tenant:1cc8a6f23e014d75bb434e4e526a41a2 on resource:security_group {{(pid=91643) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:35:06.680304 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.quota.resource [req-b12cbc34-7efc-435f-b135-8f8fbde0a55c req-d5a041aa-885e-46ff-b819-e699f4fe4c4b tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Unset dirty status for tenant:1cc8a6f23e014d75bb434e4e526a41a2 on resource:security_group {{(pid=91643) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:35:06.681692 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b12cbc34-7efc-435f-b135-8f8fbde0a55c req-d5a041aa-885e-46ff-b819-e699f4fe4c4b tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0851674 May 20 04:35:06.749335 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-de478202-6dfd-413c-815c-dee99190ad6e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:06.755200 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-26a97ac7-6748-4586-918a-8cc7336dcf66 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:06.829290 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-de478202-6dfd-413c-815c-dee99190ad6e admin admin] Pushing event deleted for resources: {'Port': ['ID=8f9c794c-3d7c-41d8-8927-b7ccdfe087ae,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:06.835431 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-de478202-6dfd-413c-815c-dee99190ad6e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:06.849481 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-da376a39-a706-42c2-b1ff-85385306abaf admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:06.870283 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-da376a39-a706-42c2-b1ff-85385306abaf admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:35:06.877081 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-da376a39-a706-42c2-b1ff-85385306abaf admin admin] Usage tracker for resource:security_group_rule and tenant:1b5d2cea82a240348302af884abab95a is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:35:06.882305 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-da376a39-a706-42c2-b1ff-85385306abaf admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:35:06.892523 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-de478202-6dfd-413c-815c-dee99190ad6e admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-8053346'] for router_gateway, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:06.898018 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-da376a39-a706-42c2-b1ff-85385306abaf admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:35:07.023390 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d {{(pid=91644) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:35:07.035862 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:07.066389 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Transition to ACTIVE for port object 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d will not be triggered until provisioned by entity L2. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:35:07.067766 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-de478202-6dfd-413c-815c-dee99190ad6e admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-8071641', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-260698'] for router, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:07.070155 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-de478202-6dfd-413c-815c-dee99190ad6e admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:07.080715 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:07.085034 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-45fe642e-321c-48de-aa9a-c4fdfe78b593 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:07.095581 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-de478202-6dfd-413c-815c-dee99190ad6e admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-8056629'] for router, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:07.096145 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-de478202-6dfd-413c-815c-dee99190ad6e admin admin] Fanout notify agent at l3_agent the message router_deleted on router 4969b258-f606-4def-b666-09764c475749 {{(pid=91643) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 20 04:35:07.109000 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-de478202-6dfd-413c-815c-dee99190ad6e admin admin] Notify callbacks [] for router, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:07.112524 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-de478202-6dfd-413c-815c-dee99190ad6e admin admin] 10.222.0.39 "DELETE /v2.0/routers/4969b258-f606-4def-b666-09764c475749 HTTP/1.1" status: 204 len: 168 time: 1.5326612 May 20 04:35:07.112679 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 45156) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:07.119042 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-da376a39-a706-42c2-b1ff-85385306abaf admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:07.134315 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-da376a39-a706-42c2-b1ff-85385306abaf admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:07.172542 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_v2 [None req-14b9e203-4ba0-49e6-88da-9087511e6f92 admin admin] Deleting subnet cdea475c-2929-4ae4-9271-87cc2fb9a378 {{(pid=91643) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 20 04:35:07.175527 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-14b9e203-4ba0-49e6-88da-9087511e6f92 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:07.228103 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-14b9e203-4ba0-49e6-88da-9087511e6f92 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-1472754'] for subnet, precommit_delete_associations {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:07.234714 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Pushing event updated for resources: {'Port': ['ID=7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:07.240467 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.155s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:07.240992 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-85330b5d-dbbf-46bf-a225-d272df7a5443 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.091s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:07.275424 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-da376a39-a706-42c2-b1ff-85385306abaf admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=1b5d2cea82a240348302af884abab95a&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6677413 May 20 04:35:07.279964 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 45160) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:07.342898 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-da376a39-a706-42c2-b1ff-85385306abaf admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=7b21606d-e7a3-4498-8483-db975079a635,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:07.347758 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-da376a39-a706-42c2-b1ff-85385306abaf admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.107s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:07.358394 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-14b9e203-4ba0-49e6-88da-9087511e6f92 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-7786687'] for subnet, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:07.373030 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Attempting to bind port 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:35:07.373440 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Attempting to bind port 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': '76847180-0644-4d57-b40f-7f63a056f8ec', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 830, 'network_id': 'bb372f0c-268d-4c71-bb09-cab162d5731a'}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:35:07.373825 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Attempting to bind port 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d on network bb372f0c-268d-4c71-bb09-cab162d5731a {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:35:07.379258 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Port 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d on network bb372f0c-268d-4c71-bb09-cab162d5731a not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:35:07.379660 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Attempting to bind port 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d on network bb372f0c-268d-4c71-bb09-cab162d5731a {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:35:07.385623 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 35, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:35:07.386040 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Checking segment: {'id': '76847180-0644-4d57-b40f-7f63a056f8ec', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 830, 'network_id': 'bb372f0c-268d-4c71-bb09-cab162d5731a'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:35:07.387094 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Bound using segment: {'id': '76847180-0644-4d57-b40f-7f63a056f8ec', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 830, 'network_id': 'bb372f0c-268d-4c71-bb09-cab162d5731a'} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:35:07.387725 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Bound port: 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '76847180-0644-4d57-b40f-7f63a056f8ec', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 830, 'network_id': 'bb372f0c-268d-4c71-bb09-cab162d5731a'}}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:35:07.388607 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:07.396176 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-54fca774-282f-452b-a884-2059567e0596 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:07.457979 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-54fca774-282f-452b-a884-2059567e0596 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:07.483144 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] For port 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d, host hv-cinder-78240, cleared binding levels {{(pid=91644) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:35:07.486628 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] For port 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d,segment=NetworkSegment(76847180-0644-4d57-b40f-7f63a056f8ec),segment_id=76847180-0644-4d57-b40f-7f63a056f8ec)] {{(pid=91644) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:35:07.492050 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:07.496720 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-bf706214-b57d-41a9-8181-e951612863a9 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:07.552059 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:07.552638 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:07.554094 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-9852a495-86f9-451a-894e-55ff269cda1f req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] 10.222.0.74 "PUT /v2.0/ports/7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d HTTP/1.1" status: 200 len: 1093 time: 1.2539325 May 20 04:35:07.554604 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '06b8328b-158c-4b35-9bc3-95b9eab9515c', 'tag': '7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:35:07.591306 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Pushing event updated for resources: {'Port': ['ID=7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d,revision_number=3']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:07.596307 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-0cb646ef-efd5-450a-a532-b65dc1aeeb4b service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.100s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:07.605444 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-54fca774-282f-452b-a884-2059567e0596 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:07.636805 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-f93df6e9-fc9c-4744-abf7-01b24bb1b3d8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:07.642287 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '06b8328b-158c-4b35-9bc3-95b9eab9515c', 'tag': '7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d', 'status': 'completed', 'code': 200} May 20 04:35:07.658766 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-54fca774-282f-452b-a884-2059567e0596 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=7b21606d-e7a3-4498-8483-db975079a635,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:07.665658 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-54fca774-282f-452b-a884-2059567e0596 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.029s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:07.669123 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-54fca774-282f-452b-a884-2059567e0596 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:07.671929 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-54fca774-282f-452b-a884-2059567e0596 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/7b21606d-e7a3-4498-8483-db975079a635 HTTP/1.1" status: 204 len: 168 time: 0.3892391 May 20 04:35:07.684104 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9852a495-86f9-451a-894e-55ff269cda1f req-248974c6-f36f-4d36-96b8-78b364d56895 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:07.685810 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-9852a495-86f9-451a-894e-55ff269cda1f req-248974c6-f36f-4d36-96b8-78b364d56895 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=1cc8a6f23e014d75bb434e4e526a41a2&device_id=06b8328b-158c-4b35-9bc3-95b9eab9515c HTTP/1.1" status: 200 len: 1096 time: 0.0811727 May 20 04:35:07.703561 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-14b9e203-4ba0-49e6-88da-9087511e6f92 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-7786632', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2050499', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2172490'] for subnet, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:07.707785 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-2720e24f-e6ee-4831-9dc1-edc5f7bc2800 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:07.746270 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-14b9e203-4ba0-49e6-88da-9087511e6f92 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=cdea475c-2929-4ae4-9271-87cc2fb9a378,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:07.749352 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-14b9e203-4ba0-49e6-88da-9087511e6f92 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.042s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:07.750339 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-14b9e203-4ba0-49e6-88da-9087511e6f92 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:07.756716 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-14b9e203-4ba0-49e6-88da-9087511e6f92 admin admin] 10.222.0.39 "DELETE /v2.0/subnets/cdea475c-2929-4ae4-9271-87cc2fb9a378 HTTP/1.1" status: 204 len: 168 time: 0.6430793 May 20 04:35:07.759291 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 45172) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:07.759679 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-05dd0cae-d8f5-4ad5-850b-fff2447b732e None None] Network 14c537db-01aa-4360-a348-47f94c75e03b requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:35:07.787465 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-9852a495-86f9-451a-894e-55ff269cda1f req-8fdf52c3-c885-40f7-a760-3a9ae1352b0f service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d HTTP/1.1" status: 200 len: 217 time: 0.0919840 May 20 04:35:07.870192 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9852a495-86f9-451a-894e-55ff269cda1f req-b4026377-a87d-4723-b12c-59b43a1ab867 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:07.871682 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-9852a495-86f9-451a-894e-55ff269cda1f req-b4026377-a87d-4723-b12c-59b43a1ab867 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=953afe7b-ba49-4dd0-b9aa-085c509e22c4 HTTP/1.1" status: 200 len: 854 time: 0.0791616 May 20 04:35:07.932566 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-9852a495-86f9-451a-894e-55ff269cda1f req-7020de7d-187d-467a-aefb-52541d00b176 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=bb372f0c-268d-4c71-bb09-cab162d5731a&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0531845 May 20 04:35:07.934629 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-593607'] for network, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:08.007206 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d1f3d9b8-7690-40a2-becb-9f28bb61c33c req-fdab013f-4061-445f-af7f-ac170596bf18 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:08.008727 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d1f3d9b8-7690-40a2-becb-9f28bb61c33c req-fdab013f-4061-445f-af7f-ac170596bf18 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=06b8328b-158c-4b35-9bc3-95b9eab9515c HTTP/1.1" status: 200 len: 943 time: 0.1572461 May 20 04:35:08.059308 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] DHCP port deletion for 14c537db-01aa-4360-a348-47f94c75e03b request from dv-cinder-782405-9-iscsi {{(pid=91645) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 20 04:35:08.067201 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] Deleting port cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:35:08.067564 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:08.100934 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-9852a495-86f9-451a-894e-55ff269cda1f req-f6bd6845-0634-4746-9c36-edf79fd4ddb3 service neutron] 10.222.0.74 "GET /v2.0/networks/bb372f0c-268d-4c71-bb09-cab162d5731a?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1607177 May 20 04:35:08.155080 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] Deleting port cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae {{(pid=91645) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:35:08.155877 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:08.227518 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d1f3d9b8-7690-40a2-becb-9f28bb61c33c req-1af5535b-766a-45ef-b32c-482d87b2a08b tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2090321 May 20 04:35:08.270814 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-9852a495-86f9-451a-894e-55ff269cda1f req-863e4835-a414-4212-840f-7aecc1640860 service neutron] 10.222.0.74 "GET /v2.0/networks/bb372f0c-268d-4c71-bb09-cab162d5731a?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1641023 May 20 04:35:08.315543 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:35:08.318911 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] For port cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:35:08.319566 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:08.366056 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:35:08.366404 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 8c2dbf0c-35ac-4b3b-90ce-24bafe9dadb7 not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:35:08.366820 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 8f9c794c-3d7c-41d8-8927-b7ccdfe087ae no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:35:08.406616 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: 8f9c794c-3d7c-41d8-8927-b7ccdfe087ae {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:35:08.406962 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 8f9c794c-3d7c-41d8-8927-b7ccdfe087ae not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:35:08.407414 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:35:08.411523 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0acc9dce-f9c1-4a55-bdb9-407dffcaf6dd req-aa3885ba-aedd-4253-adfb-6b4b1326eb4d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:08.413375 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0acc9dce-f9c1-4a55-bdb9-407dffcaf6dd req-aa3885ba-aedd-4253-adfb-6b4b1326eb4d service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=1cc8a6f23e014d75bb434e4e526a41a2&device_id=06b8328b-158c-4b35-9bc3-95b9eab9515c HTTP/1.1" status: 200 len: 1096 time: 0.0972974 May 20 04:35:08.467189 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] Calling delete_port for cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae owned by network:dhcp {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:35:08.469203 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] For port cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae,segment=NetworkSegment(05406cc0-328f-4fba-b043-20b2b57b274d),segment_id=05406cc0-328f-4fba-b043-20b2b57b274d)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:35:08.469601 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:08.534229 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:08.541197 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-8329845e-83ae-4406-9a8d-df65cf02c55f tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:08.580917 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0acc9dce-f9c1-4a55-bdb9-407dffcaf6dd req-3dffaaa1-aff8-4581-883c-f9a137f8f287 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:08.582264 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0acc9dce-f9c1-4a55-bdb9-407dffcaf6dd req-3dffaaa1-aff8-4581-883c-f9a137f8f287 service neutron] 10.222.0.74 "GET /v2.0/networks?id=bb372f0c-268d-4c71-bb09-cab162d5731a HTTP/1.1" status: 200 len: 915 time: 0.1617231 May 20 04:35:08.592385 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] Pushing event deleted for resources: {'Port': ['ID=cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:08.596972 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.056s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:08.600466 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] No DVR routers for this DVR port cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae on host dv-cinder-782405-9-iscsi {{(pid=91643) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:35:08.621459 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] Calling delete_port for cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae owned by network:dhcp {{(pid=91645) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:35:08.624075 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Port cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae update to DOWN by agent not found {{(pid=91645) update_port_statuses /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2123}} May 20 04:35:08.672637 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716'] for port, after_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:08.677618 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-eb472546-3579-475a-8577-8f521cc75f1f None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:08.684247 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0acc9dce-f9c1-4a55-bdb9-407dffcaf6dd req-27297982-3194-4e0d-998b-0f5caec19a7d service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d HTTP/1.1" status: 200 len: 217 time: 0.0943217 May 20 04:35:08.723509 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] Pushing event deleted for resources: {'Port': ['ID=cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae,revision_number=None']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:08.734429 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:08.735069 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:08.735487 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:08.735876 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:08.736259 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:08.736778 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:08.737432 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:08.737559 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:08.737930 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:08.738313 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:08.738687 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:08.739458 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:08.741486 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-7789108', 'neutron.services.segments.db._delete_segments_for_network-8760688958082'] for network, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:08.745324 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.066s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:08.747044 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3ddde11e-5f3d-4925-84b5-0afd971e9f44 None None] No DVR routers for this DVR port cbc79e7a-8d9c-4eb7-aa9c-f3b0d08b60ae on host dv-cinder-782405-9-iscsi {{(pid=91645) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:35:08.760513 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['14c537db-01aa-4360-a348-47f94c75e03b']}} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:35:08.768271 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0acc9dce-f9c1-4a55-bdb9-407dffcaf6dd req-14704069-ed22-4dc3-87b2-5168d1659f66 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:08.769717 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0acc9dce-f9c1-4a55-bdb9-407dffcaf6dd req-14704069-ed22-4dc3-87b2-5168d1659f66 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=953afe7b-ba49-4dd0-b9aa-085c509e22c4 HTTP/1.1" status: 200 len: 854 time: 0.0782874 May 20 04:35:08.774842 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '05406cc0-328f-4fba-b043-20b2b57b274d') {'for_net_delete': True} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:35:08.775138 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '05406cc0-328f-4fba-b043-20b2b57b274d') {} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:35:08.787311 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8760688968331', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-1472741'] for segment, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:08.804929 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:08.812020 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] Releasing vlan 841 on physical network physnet1 to pool {{(pid=91643) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 20 04:35:08.812453 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2070625'] for segment, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: Traceback (most recent call last): May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: timer() May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: cb(*args, **kw) May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: return func(*args, **kwargs) May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: self._notify() May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: self.callback(batched_events) May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: event.method(event) May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: aggregate_id = self._get_aggregate_id(event.segment_id) May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: aggregate_uuid = self.p_client.list_aggregates( May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: return f(self, *a, **k) May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: return self._get(url).json() May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: return self._client.get(url, endpoint_filter=self._ks_filter, May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: return self.request(url, 'GET', **kwargs) May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: base_url = self.get_endpoint(auth, allow=allow, May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: auth = self._auth_required(auth, 'determine endpoint URL') May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 20 04:35:08.830848 dv-cinder-782405-9-iscsi neutron-server[91643]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 20 04:35:08.908766 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-7789117', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-8053321'] for network, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:08.916512 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-634c52ba-d57f-490c-acf3-4ea6f3ea0ba7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:08.924135 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0acc9dce-f9c1-4a55-bdb9-407dffcaf6dd req-f34dad4f-b867-4179-b23d-9ba7a68c0a98 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=bb372f0c-268d-4c71-bb09-cab162d5731a&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.1471705 May 20 04:35:08.931964 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:08.949158 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] Pushing event deleted for resources: {'Network': ['ID=14c537db-01aa-4360-a348-47f94c75e03b,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:08.953605 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.041s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:08.955171 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-9e369c5b-3f7e-4e04-99bf-84da05a8c15a admin admin] 10.222.0.39 "DELETE /v2.0/networks/14c537db-01aa-4360-a348-47f94c75e03b HTTP/1.1" status: 204 len: 168 time: 1.1946552 May 20 04:35:08.958066 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 45200) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:09.060105 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0acc9dce-f9c1-4a55-bdb9-407dffcaf6dd req-03398ba1-eb67-4fa3-a647-674e87b6ec30 service neutron] 10.222.0.74 "GET /v2.0/networks/bb372f0c-268d-4c71-bb09-cab162d5731a?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1289268 May 20 04:35:09.220116 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-0acc9dce-f9c1-4a55-bdb9-407dffcaf6dd req-039bfd61-be88-4c3e-b545-01d388027eb5 service neutron] 10.222.0.74 "GET /v2.0/networks/bb372f0c-268d-4c71-bb09-cab162d5731a?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1519830 May 20 04:35:09.490763 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-624072af-2602-42f9-b546-3ded8e6f4325 req-76d944b0-cb43-44ae-aeec-5959b3263bf1 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:09.492113 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-624072af-2602-42f9-b546-3ded8e6f4325 req-76d944b0-cb43-44ae-aeec-5959b3263bf1 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=06b8328b-158c-4b35-9bc3-95b9eab9515c HTTP/1.1" status: 200 len: 943 time: 0.1448195 May 20 04:35:09.544196 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-624072af-2602-42f9-b546-3ded8e6f4325 req-c5aaf864-6cf9-4fc6-b949-8ecb76d60d15 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0461159 May 20 04:35:09.612112 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6507ec19-4d4f-4e50-b63c-40a031800c7c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-8003145'] for router_interface, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:10.120926 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-6507ec19-4d4f-4e50-b63c-40a031800c7c admin admin] Deleting port aedd7af1-ba1a-486f-a289-b5f1d46d3999 {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:35:10.121264 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6507ec19-4d4f-4e50-b63c-40a031800c7c admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:10.244876 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [None req-6507ec19-4d4f-4e50-b63c-40a031800c7c admin admin] For port aedd7af1-ba1a-486f-a289-b5f1d46d3999, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=aedd7af1-ba1a-486f-a289-b5f1d46d3999,segment=NetworkSegment(3bd4c975-0dc6-43e6-b17a-8e2fa22a9507),segment_id=3bd4c975-0dc6-43e6-b17a-8e2fa22a9507)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:35:10.245130 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6507ec19-4d4f-4e50-b63c-40a031800c7c admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:10.366588 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-6507ec19-4d4f-4e50-b63c-40a031800c7c admin admin] Calling delete_port for aedd7af1-ba1a-486f-a289-b5f1d46d3999 owned by network:router_interface {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:35:10.513308 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6507ec19-4d4f-4e50-b63c-40a031800c7c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:10.516925 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-bf4ea990-2fac-4b27-9942-558b38f25d08 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:10.561456 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6507ec19-4d4f-4e50-b63c-40a031800c7c admin admin] Pushing event deleted for resources: {'Port': ['ID=aedd7af1-ba1a-486f-a289-b5f1d46d3999,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:10.602065 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-6507ec19-4d4f-4e50-b63c-40a031800c7c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:10.909850 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fe14cc35-c913-405a-9f63-525a100e8e26 req-8f86ba2c-d21a-4e54-9a7e-6fbaa88d69e2 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:10.911309 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fe14cc35-c913-405a-9f63-525a100e8e26 req-8f86ba2c-d21a-4e54-9a7e-6fbaa88d69e2 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=06b8328b-158c-4b35-9bc3-95b9eab9515c HTTP/1.1" status: 200 len: 943 time: 0.2509997 May 20 04:35:10.913653 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-6507ec19-4d4f-4e50-b63c-40a031800c7c admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-7924922', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-8054745'] for router_interface, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:11.043594 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-fe14cc35-c913-405a-9f63-525a100e8e26 req-478cd8ec-841f-4ddb-b811-7d6c07948013 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1259561 May 20 04:35:11.434858 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-6507ec19-4d4f-4e50-b63c-40a031800c7c admin admin] Router 5faa3599-06bc-45d7-8b3f-e21d104b087a has already been hosted by L3 agent 3cbc3a5c-3694-4437-860c-9de9197240a6 {{(pid=91643) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 20 04:35:11.616272 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-6507ec19-4d4f-4e50-b63c-40a031800c7c admin admin] Notify agent at l3_agent.dv-cinder-782405-9-iscsi the message routers_updated {{(pid=91643) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 20 04:35:11.627300 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-33a55508-421c-4087-a23b-45ba30cbe69c None None] Sync routers for ids ['5faa3599-06bc-45d7-8b3f-e21d104b087a'] in dv-cinder-782405-9-iscsi {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 20 04:35:11.628681 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-6507ec19-4d4f-4e50-b63c-40a031800c7c admin admin] 10.222.0.39 "PUT /v2.0/routers/5faa3599-06bc-45d7-8b3f-e21d104b087a/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.6693473 May 20 04:35:11.631934 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 45234) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:11.640867 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-33a55508-421c-4087-a23b-45ba30cbe69c None None] Routers {'5faa3599-06bc-45d7-8b3f-e21d104b087a'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:35:11.641477 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-33a55508-421c-4087-a23b-45ba30cbe69c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:34:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5faa3599-06bc-45d7-8b3f-e21d104b087a'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:35:11.661434 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:35:11.804759 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d,segment=NetworkSegment(76847180-0644-4d57-b40f-7f63a056f8ec),segment_id=76847180-0644-4d57-b40f-7f63a056f8ec)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:35:11.809180 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Notify callbacks [] for router, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:11.818613 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': '7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d', 'network_id': 'bb372f0c-268d-4c71-bb09-cab162d5731a', 'port_id': '7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d', 'mac_address': 'fa:16:3e:ac:7e:31', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 830, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '953afe7b-ba49-4dd0-b9aa-085c509e22c4', 'ip_address': '172.20.0.9'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:35:11.819854 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-33a55508-421c-4087-a23b-45ba30cbe69c None None] Set router 5faa3599-06bc-45d7-8b3f-e21d104b087a gateway port host: dv-cinder-782405-9-iscsi {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:35:11.874285 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:11.892196 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:35:11.909128 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d,segment=NetworkSegment(76847180-0644-4d57-b40f-7f63a056f8ec),segment_id=76847180-0644-4d57-b40f-7f63a056f8ec)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:35:11.913947 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:11.915863 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-df1d3eb6-cafc-4b9c-9720-c3131933513b None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:11.986191 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:11.989795 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:11.993451 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:12.068461 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Deleting port 0e5ce97c-8370-428d-b569-67d6ffe03e3b {{(pid=91643) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:35:12.068856 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:12.115879 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-33a55508-421c-4087-a23b-45ba30cbe69c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5faa3599-06bc-45d7-8b3f-e21d104b087a', 'name': 'tempest-TestVolumeBootPattern-1346048449-router', 'tenant_id': '207fc367830344daad06243fb26256f7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'external_fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.182'}], 'enable_snat': True}, 'gw_port_id': '0e5ce97c-8370-428d-b569-67d6ffe03e3b', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:42Z', 'updated_at': '2021-05-20T04:35:10Z', 'revision_number': 5, 'project_id': '207fc367830344daad06243fb26256f7', 'gw_port': {'id': '0e5ce97c-8370-428d-b569-67d6ffe03e3b', 'name': '', 'network_id': '712ce8c4-8feb-4719-bf13-e06992125cd9', 'tenant_id': '', 'mac_address': 'fa:16:3e:76:ec:16', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '5faa3599-06bc-45d7-8b3f-e21d104b087a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 326, 'fixed_ips': [{'subnet_id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'ip_address': '172.27.0.182', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-782405-9-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-20T04:24:43Z', 'updated_at': '2021-05-20T04:24:52Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': 'e5bd8558-7bc3-4ef0-8a03-a3fb8d29d6f5', 'cidr': '172.27.0.0/24', 'gateway_ip': '172.27.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-782405-9-iscsi'}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:35:12.119797 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-33a55508-421c-4087-a23b-45ba30cbe69c None None] Checking router: 5faa3599-06bc-45d7-8b3f-e21d104b087a for host: dv-cinder-782405-9-iscsi {{(pid=91646) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 20 04:35:12.128377 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvrscheduler_db [None req-33a55508-421c-4087-a23b-45ba30cbe69c None None] Routers {'5faa3599-06bc-45d7-8b3f-e21d104b087a'} bound to L3 agent in host dv-cinder-782405-9-iscsi {{(pid=91646) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 20 04:35:12.128989 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-33a55508-421c-4087-a23b-45ba30cbe69c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-782405-9-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-20T04:14:15Z,description=,heartbeat_timestamp=2021-05-20T04:34:45Z,host='dv-cinder-782405-9-iscsi',id=3cbc3a5c-3694-4437-860c-9de9197240a6,load=0,resource_versions=,resources_synced=,started_at=2021-05-20T04:14:15Z,topic='l3_agent')) {'router_ids': ['5faa3599-06bc-45d7-8b3f-e21d104b087a'], 'active': True} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:35:12.259820 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.db [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] For port 0e5ce97c-8370-428d-b569-67d6ffe03e3b, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=0e5ce97c-8370-428d-b569-67d6ffe03e3b,segment=NetworkSegment(240fc062-60cd-4ee1-8e1d-b9956ca52b7b),segment_id=240fc062-60cd-4ee1-8e1d-b9956ca52b7b)] {{(pid=91643) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:35:12.260189 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:12.277603 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:35:12.286696 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_dvr_db [None req-33a55508-421c-4087-a23b-45ba30cbe69c None None] Set router 5faa3599-06bc-45d7-8b3f-e21d104b087a gateway port host: None {{(pid=91646) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 20 04:35:12.390567 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.plugin [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Calling delete_port for 0e5ce97c-8370-428d-b569-67d6ffe03e3b owned by network:router_gateway {{(pid=91643) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:35:12.429648 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.l3_hamode_db [None req-33a55508-421c-4087-a23b-45ba30cbe69c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '5faa3599-06bc-45d7-8b3f-e21d104b087a', 'name': 'tempest-TestVolumeBootPattern-1346048449-router', 'tenant_id': '207fc367830344daad06243fb26256f7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:42Z', 'updated_at': '2021-05-20T04:35:11Z', 'revision_number': 6, 'project_id': '207fc367830344daad06243fb26256f7', 'gw_port_host': None}], 'dv-cinder-782405-9-iscsi', False) {} {{(pid=91646) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:35:12.441732 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-33a55508-421c-4087-a23b-45ba30cbe69c None None] The sync data for ids ['5faa3599-06bc-45d7-8b3f-e21d104b087a'] in dv-cinder-782405-9-iscsi is: [{'id': '5faa3599-06bc-45d7-8b3f-e21d104b087a', 'name': 'tempest-TestVolumeBootPattern-1346048449-router', 'tenant_id': '207fc367830344daad06243fb26256f7', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-20T04:24:42Z', 'updated_at': '2021-05-20T04:35:11Z', 'revision_number': 6, 'project_id': '207fc367830344daad06243fb26256f7', 'gw_port_host': None}] {{(pid=91646) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 20 04:35:12.469514 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c24c209a-a825-4eff-a639-273469756e17 req-06bd0f83-2a85-4afc-bad0-af0f1f3d3e6d tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:12.471027 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c24c209a-a825-4eff-a639-273469756e17 req-06bd0f83-2a85-4afc-bad0-af0f1f3d3e6d tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=06b8328b-158c-4b35-9bc3-95b9eab9515c HTTP/1.1" status: 200 len: 944 time: 0.2907569 May 20 04:35:12.546655 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-c24c209a-a825-4eff-a639-273469756e17 req-c80c0ef2-1f98-4e8e-beb3-0d517be82aa3 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0632370 May 20 04:35:12.609358 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2070556'] for port, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:12.614483 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-40628d21-1644-4afa-8dc7-2837ad9834b7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:12.624698 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-510d9967-13d2-48cc-a3a5-bc5fd259839e None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:12.693981 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Pushing event deleted for resources: {'Port': ['ID=0e5ce97c-8370-428d-b569-67d6ffe03e3b,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:12.712523 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:12.712909 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:12.713293 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:12.713659 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:12.714019 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:12.714383 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:12.714746 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:12.715234 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:12.715623 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:12.715980 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:12.716340 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:12.717463 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:35:12.747633 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.133s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:12.776849 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-8053346'] for router_gateway, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:13.063014 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-8071641', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-260698'] for router, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:13.065400 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:13.095467 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-8056629'] for router, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:13.096014 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Fanout notify agent at l3_agent the message router_deleted on router 5faa3599-06bc-45d7-8b3f-e21d104b087a {{(pid=91643) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 20 04:35:13.108807 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] Notify callbacks [] for router, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:13.109920 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-f96378c0-4aa2-44c7-8c00-e6aea4e01667 admin admin] 10.222.0.39 "DELETE /v2.0/routers/5faa3599-06bc-45d7-8b3f-e21d104b087a HTTP/1.1" status: 204 len: 168 time: 1.4770963 May 20 04:35:13.112718 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 45248) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:13.172210 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.db.db_base_plugin_v2 [None req-30ede2b3-2068-4a6a-90a0-9d5dab656d2f admin admin] Deleting subnet 2927e09a-9033-4dd7-a313-77a3b77cd585 {{(pid=91643) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 20 04:35:13.175394 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-30ede2b3-2068-4a6a-90a0-9d5dab656d2f admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:13.229464 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-30ede2b3-2068-4a6a-90a0-9d5dab656d2f admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-1472754'] for subnet, precommit_delete_associations {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:13.312390 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-30ede2b3-2068-4a6a-90a0-9d5dab656d2f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-7786687'] for subnet, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:13.566706 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-30ede2b3-2068-4a6a-90a0-9d5dab656d2f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-7786632', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-2050499', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2172490'] for subnet, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:13.570585 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:13.610735 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-30ede2b3-2068-4a6a-90a0-9d5dab656d2f admin admin] Pushing event deleted for resources: {'Subnet': ['ID=2927e09a-9033-4dd7-a313-77a3b77cd585,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:13.614052 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-30ede2b3-2068-4a6a-90a0-9d5dab656d2f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:13.622723 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-c9ec3b6c-b5e9-4c6e-b284-56f5d3d710f3 None None] Network b5c87200-df2c-432b-8fc4-24090c6d30f3 requested from dv-cinder-782405-9-iscsi {{(pid=91645) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 20 04:35:13.624362 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-30ede2b3-2068-4a6a-90a0-9d5dab656d2f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for subnet, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:13.632951 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-30ede2b3-2068-4a6a-90a0-9d5dab656d2f admin admin] 10.222.0.39 "DELETE /v2.0/subnets/2927e09a-9033-4dd7-a313-77a3b77cd585 HTTP/1.1" status: 204 len: 168 time: 0.5170107 May 20 04:35:13.633739 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 45256) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:13.725816 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:35:13.748001 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8f9c027f-d86f-4665-93b6-e6015b0abf73 req-564c3ec2-a8f3-46b5-b686-986f513eda3e tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:13.758594 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8f9c027f-d86f-4665-93b6-e6015b0abf73 req-564c3ec2-a8f3-46b5-b686-986f513eda3e tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=06b8328b-158c-4b35-9bc3-95b9eab9515c HTTP/1.1" status: 200 len: 944 time: 0.1008282 May 20 04:35:13.768571 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-593607'] for network, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:13.801895 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8f9c027f-d86f-4665-93b6-e6015b0abf73 req-ec299247-cd7d-43c5-8804-8b2d273ac76f tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0423748 May 20 04:35:13.857984 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] Deleting port 83f0181b-202d-4c5d-ab40-d746ebe28872 {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:35:13.858321 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:13.884109 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:35:13.891654 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:35:13.892027 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:13.962306 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] For port 83f0181b-202d-4c5d-ab40-d746ebe28872, host dv-cinder-782405-9-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-782405-9-iscsi',level=0,port_id=83f0181b-202d-4c5d-ab40-d746ebe28872,segment=NetworkSegment(3bd4c975-0dc6-43e6-b17a-8e2fa22a9507),segment_id=3bd4c975-0dc6-43e6-b17a-8e2fa22a9507)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:35:13.963442 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:14.003707 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-cfa60d18-4cc7-44eb-bce6-6e088f2c9e9b None None] DHCP port deletion for b5c87200-df2c-432b-8fc4-24090c6d30f3 request from dv-cinder-782405-9-iscsi {{(pid=91645) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 20 04:35:14.058544 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:14.095775 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] Calling delete_port for 83f0181b-202d-4c5d-ab40-d746ebe28872 owned by network:dhcp {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:35:14.096019 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '06b8328b-158c-4b35-9bc3-95b9eab9515c', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:35:14.108792 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d,segment=NetworkSegment(76847180-0644-4d57-b40f-7f63a056f8ec),segment_id=76847180-0644-4d57-b40f-7f63a056f8ec)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:35:14.116844 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:14.119298 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:14.156441 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-cfa60d18-4cc7-44eb-bce6-6e088f2c9e9b None None] Deleting port 83f0181b-202d-4c5d-ab40-d746ebe28872 {{(pid=91645) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:35:14.156656 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-cfa60d18-4cc7-44eb-bce6-6e088f2c9e9b None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:14.161027 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:14.166942 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-bf01a353-e42e-4716-8d2e-032e245cb761 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:14.222128 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] Pushing event deleted for resources: {'Port': ['ID=83f0181b-202d-4c5d-ab40-d746ebe28872,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:14.224094 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] No DVR routers for this DVR port 83f0181b-202d-4c5d-ab40-d746ebe28872 on host dv-cinder-782405-9-iscsi {{(pid=91644) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:35:14.225730 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.059s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:14.256375 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 83f0181b-202d-4c5d-ab40-d746ebe28872 no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:35:14.301814 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: 83f0181b-202d-4c5d-ab40-d746ebe28872 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:35:14.302004 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 83f0181b-202d-4c5d-ab40-d746ebe28872 not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:35:14.302386 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device aedd7af1-ba1a-486f-a289-b5f1d46d3999 no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:35:14.326607 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '06b8328b-158c-4b35-9bc3-95b9eab9515c', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d', 'code': 200} May 20 04:35:14.360259 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-7789108', 'neutron.services.segments.db._delete_segments_for_network-8760688958082'] for network, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:14.374409 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['b5c87200-df2c-432b-8fc4-24090c6d30f3']}} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:35:14.390357 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '3bd4c975-0dc6-43e6-b17a-8e2fa22a9507') {'for_net_delete': True} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:35:14.390718 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.services.segments.db [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '3bd4c975-0dc6-43e6-b17a-8e2fa22a9507') {} {{(pid=91644) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:35:14.401121 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:14.403024 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8760688968331', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-853278'] for segment, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:14.406628 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: aedd7af1-ba1a-486f-a289-b5f1d46d3999 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:35:14.407168 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device aedd7af1-ba1a-486f-a289-b5f1d46d3999 not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:35:14.407790 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 0e5ce97c-8370-428d-b569-67d6ffe03e3b no longer exists at agent ovs-agent-dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:35:14.420810 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:14.429579 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] Releasing vlan 810 on physical network physnet1 to pool {{(pid=91644) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 20 04:35:14.430154 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2001055'] for segment, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:14.433537 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.314s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: Traceback (most recent call last): May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: timer() May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: cb(*args, **kw) May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: return func(*args, **kwargs) May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: self._notify() May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: self.callback(batched_events) May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: event.method(event) May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: aggregate_id = self._get_aggregate_id(event.segment_id) May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: aggregate_uuid = self.p_client.list_aggregates( May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: return f(self, *a, **k) May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: return self._get(url).json() May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: return self._client.get(url, endpoint_filter=self._ks_filter, May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: return self.request(url, 'GET', **kwargs) May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: base_url = self.get_endpoint(auth, allow=allow, May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: auth = self._auth_required(auth, 'determine endpoint URL') May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 20 04:35:14.445020 dv-cinder-782405-9-iscsi neutron-server[91644]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 20 04:35:14.458356 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-198357cd-aaad-4397-a100-ced747a19175 None None] No Port match for: 0e5ce97c-8370-428d-b569-67d6ffe03e3b {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:35:14.459080 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Device 0e5ce97c-8370-428d-b569-67d6ffe03e3b not bound to the agent host dv-cinder-782405-9-iscsi {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:35:14.465015 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-cfa60d18-4cc7-44eb-bce6-6e088f2c9e9b None None] For port 83f0181b-202d-4c5d-ab40-d746ebe28872, host dv-cinder-782405-9-iscsi, got binding levels [] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:35:14.465609 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-cfa60d18-4cc7-44eb-bce6-6e088f2c9e9b None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:14.587455 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-7789117', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-8053321'] for network, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:14.594315 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:14.597106 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-cfa60d18-4cc7-44eb-bce6-6e088f2c9e9b None None] Calling delete_port for 83f0181b-202d-4c5d-ab40-d746ebe28872 owned by network:dhcp {{(pid=91645) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:35:14.600656 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.ipam_backend_mixin [None req-cfa60d18-4cc7-44eb-bce6-6e088f2c9e9b None None] Port 83f0181b-202d-4c5d-ab40-d746ebe28872 was not found and therefore no delete operation was performed {{(pid=91645) delete_port /opt/stack/neutron/neutron/db/ipam_backend_mixin.py:566}} May 20 04:35:14.601868 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-cfa60d18-4cc7-44eb-bce6-6e088f2c9e9b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716'] for port, after_delete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:14.606090 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:14.609301 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:14.620138 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] Pushing event deleted for resources: {'Network': ['ID=b5c87200-df2c-432b-8fc4-24090c6d30f3,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:14.625060 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.031s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:14.635545 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-eaf5af1f-308f-4042-9845-22d5e2ca5655 admin admin] 10.222.0.39 "DELETE /v2.0/networks/b5c87200-df2c-432b-8fc4-24090c6d30f3 HTTP/1.1" status: 204 len: 168 time: 1.0001178 May 20 04:35:14.638610 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-011cc510-a8f9-46dc-bbe9-09118630bd48 req-a7eb99fc-b5eb-44d3-9b02-d8511b20e60b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:14.640008 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-011cc510-a8f9-46dc-bbe9-09118630bd48 req-a7eb99fc-b5eb-44d3-9b02-d8511b20e60b service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=06b8328b-158c-4b35-9bc3-95b9eab9515c HTTP/1.1" status: 200 len: 1098 time: 0.1202898 May 20 04:35:14.642880 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api [None req-cfa60d18-4cc7-44eb-bce6-6e088f2c9e9b None None] Network b5c87200-df2c-432b-8fc4-24090c6d30f3 is not hosted by any dhcp agent {{(pid=91645) _notify_agents /opt/stack/neutron/neutron/api/rpc/agentnotifiers/dhcp_rpc_agent_api.py:231}} May 20 04:35:14.643387 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.l3_dvrscheduler_db [None req-cfa60d18-4cc7-44eb-bce6-6e088f2c9e9b None None] No DVR routers for this DVR port 83f0181b-202d-4c5d-ab40-d746ebe28872 on host dv-cinder-782405-9-iscsi {{(pid=91645) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:35:14.653344 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cfa60d18-4cc7-44eb-bce6-6e088f2c9e9b None None] Pushing event deleted for resources: {'Port': ['ID=83f0181b-202d-4c5d-ab40-d746ebe28872,revision_number=None']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:14.655586 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-cfa60d18-4cc7-44eb-bce6-6e088f2c9e9b None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:14.685822 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-011cc510-a8f9-46dc-bbe9-09118630bd48 req-ef6e9482-ceb1-4996-a96e-7f32180334ce service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0380299 May 20 04:35:14.852097 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 45276) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:14.901750 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-c238e46f-dbc2-46f7-bb46-f0ebbe13c6ca admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=0e3baa0ea24d42f3ab59e40845b350fd&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0483465 May 20 04:35:14.904116 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 45282) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:14.971147 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:14.976670 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e8ccd585-f2b4-400b-9970-76c9bc778c85 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:15.004277 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e8ccd585-f2b4-400b-9970-76c9bc778c85 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:15.006576 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-424aaa90-d13a-46e1-a378-1986f6522b53 req-47613d53-361e-4d3f-9c1a-19e3777bd7b9 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:15.008192 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-424aaa90-d13a-46e1-a378-1986f6522b53 req-47613d53-361e-4d3f-9c1a-19e3777bd7b9 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=06b8328b-158c-4b35-9bc3-95b9eab9515c HTTP/1.1" status: 200 len: 945 time: 0.0974069 May 20 04:35:15.063879 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-424aaa90-d13a-46e1-a378-1986f6522b53 req-7a12a4fe-d25e-40ad-ac04-8cb0fe432f74 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0497832 May 20 04:35:15.065817 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e8ccd585-f2b4-400b-9970-76c9bc778c85 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:15.075885 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-3b45077f-5266-44df-93ef-32da6627c10d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:15.086249 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e8ccd585-f2b4-400b-9970-76c9bc778c85 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=9f4aef68-406c-46a5-9e8f-df01ef8e502f,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:15.090380 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-e8ccd585-f2b4-400b-9970-76c9bc778c85 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:15.098075 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-e8ccd585-f2b4-400b-9970-76c9bc778c85 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:15.102098 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-e8ccd585-f2b4-400b-9970-76c9bc778c85 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/9f4aef68-406c-46a5-9e8f-df01ef8e502f HTTP/1.1" status: 204 len: 168 time: 0.1940041 May 20 04:35:15.121921 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-a0cc448f-d570-4a91-bd6d-899d54864c75 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:15.495643 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 45294) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:15.548659 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-d1209eaa-2ff7-40bb-8f80-6fa3ca6d9315 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=207fc367830344daad06243fb26256f7&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0517104 May 20 04:35:15.550930 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 45298) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:35:15.608912 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-984dd60d-482b-4ad8-b3d4-fb2fb48e704e admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:15.634655 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-984dd60d-482b-4ad8-b3d4-fb2fb48e704e admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:15.687653 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-984dd60d-482b-4ad8-b3d4-fb2fb48e704e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:15.692739 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-8d090719-4bab-4d4c-8e33-9a9bbba18c5e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:35:15.704305 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-984dd60d-482b-4ad8-b3d4-fb2fb48e704e admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=66fa43f5-dfa4-437d-b177-e6992999a970,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:35:15.707296 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-984dd60d-482b-4ad8-b3d4-fb2fb48e704e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:35:15.710292 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-984dd60d-482b-4ad8-b3d4-fb2fb48e704e admin admin] Notify callbacks [] for security_group, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:15.710483 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-984dd60d-482b-4ad8-b3d4-fb2fb48e704e admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/66fa43f5-dfa4-437d-b177-e6992999a970 HTTP/1.1" status: 204 len: 168 time: 0.1585267 May 20 04:35:16.161774 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7fe44aa1-7bcc-42d1-b913-a861ea4e125b req-74b02c4b-54fb-446d-ac33-c24dd548ed1d service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=06b8328b-158c-4b35-9bc3-95b9eab9515c&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.1804245 May 20 04:35:16.240693 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:16.255585 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1cc659ec-9130-412f-ab8a-58d833c8a2f8 req-660e687a-1b87-4507-8211-a8ab64af9103 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:16.257174 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1cc659ec-9130-412f-ab8a-58d833c8a2f8 req-660e687a-1b87-4507-8211-a8ab64af9103 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=06b8328b-158c-4b35-9bc3-95b9eab9515c HTTP/1.1" status: 200 len: 945 time: 0.0892544 May 20 04:35:16.299766 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7fe44aa1-7bcc-42d1-b913-a861ea4e125b req-381297bc-a885-487e-8e7c-03e807a22752 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:16.301294 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7fe44aa1-7bcc-42d1-b913-a861ea4e125b req-381297bc-a885-487e-8e7c-03e807a22752 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=1cc8a6f23e014d75bb434e4e526a41a2&device_id=06b8328b-158c-4b35-9bc3-95b9eab9515c HTTP/1.1" status: 200 len: 1098 time: 0.0717597 May 20 04:35:16.302763 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1cc659ec-9130-412f-ab8a-58d833c8a2f8 req-b9a018ea-e8ae-4b36-a493-5aec101d0b3f tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0394995 May 20 04:35:16.453593 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7fe44aa1-7bcc-42d1-b913-a861ea4e125b req-3cec6f47-7307-44e8-b10a-49aa718775b4 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:16.455264 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7fe44aa1-7bcc-42d1-b913-a861ea4e125b req-3cec6f47-7307-44e8-b10a-49aa718775b4 service neutron] 10.222.0.74 "GET /v2.0/networks?id=bb372f0c-268d-4c71-bb09-cab162d5731a HTTP/1.1" status: 200 len: 915 time: 0.1351621 May 20 04:35:16.530981 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7fe44aa1-7bcc-42d1-b913-a861ea4e125b req-f0ef3d72-eadd-4624-a422-028a17955fd2 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d HTTP/1.1" status: 200 len: 217 time: 0.0693271 May 20 04:35:16.624969 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7fe44aa1-7bcc-42d1-b913-a861ea4e125b req-88a15178-403c-4ac1-a337-e44f821c7928 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:35:16.627355 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7fe44aa1-7bcc-42d1-b913-a861ea4e125b req-88a15178-403c-4ac1-a337-e44f821c7928 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=953afe7b-ba49-4dd0-b9aa-085c509e22c4 HTTP/1.1" status: 200 len: 854 time: 0.0897505 May 20 04:35:16.677343 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7fe44aa1-7bcc-42d1-b913-a861ea4e125b req-75a987ab-e47f-49dd-90c2-f2399521b112 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=bb372f0c-268d-4c71-bb09-cab162d5731a&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0435908 May 20 04:35:16.819602 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7fe44aa1-7bcc-42d1-b913-a861ea4e125b req-1c8d7fa0-f328-4982-82c6-29ac883ea348 service neutron] 10.222.0.74 "GET /v2.0/networks/bb372f0c-268d-4c71-bb09-cab162d5731a?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1342988 May 20 04:35:17.005637 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7fe44aa1-7bcc-42d1-b913-a861ea4e125b req-4e008c97-1ddc-4e62-afe9-ad5870f52978 service neutron] 10.222.0.74 "GET /v2.0/networks/bb372f0c-268d-4c71-bb09-cab162d5731a?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1757042 May 20 04:35:17.878851 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:35.721069 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:37.023099 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:35:42.636791 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-146d61a8-497a-4b4b-b521-4e00a4d4b9b3 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:44.978876 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:45.105601 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0179d3dd-8f15-4bdd-97e0-56c76c2a5508 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:46.245808 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:35:47.886665 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:05.717086 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:10.970363 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 45734) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:36:11.109764 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Notify callbacks [] for security_group, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:11.125863 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:36:11.129930 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Usage tracker for resource:security_group_rule and tenant:cab277c32d8649c986cd5b7a4061ea66 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:36:11.134305 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:36:11.143514 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:36:11.228412 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:11.233522 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:11.241793 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-ae6a4131-2f52-4044-af50-f174d2bbd876 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:36:11.282296 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=cab277c32d8649c986cd5b7a4061ea66&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3106747 May 20 04:36:11.283953 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=403d5018-b264-4959-b6a1-f893fdc18927,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:36:11.285035 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 45740) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:36:11.292101 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:11.292488 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:11.292859 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:11.293457 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:11.293863 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:11.294332 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:11.294763 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:11.295228 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:11.295662 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:11.296069 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:11.296489 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:11.296932 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:11.301562 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-a6092f98-8984-402a-bca3-fb957efdf064 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.060s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:36:11.346318 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a60fe7eb-9167-433c-b97a-96af448079bd admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:11.372264 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a60fe7eb-9167-433c-b97a-96af448079bd admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:11.425783 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a60fe7eb-9167-433c-b97a-96af448079bd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:11.429923 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-8f46111c-7a46-4921-894c-48d10490c6a1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:36:11.440741 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a60fe7eb-9167-433c-b97a-96af448079bd admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=403d5018-b264-4959-b6a1-f893fdc18927,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:36:11.443284 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-a60fe7eb-9167-433c-b97a-96af448079bd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.013s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:36:11.448594 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-a60fe7eb-9167-433c-b97a-96af448079bd admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:11.449767 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-a60fe7eb-9167-433c-b97a-96af448079bd admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/403d5018-b264-4959-b6a1-f893fdc18927 HTTP/1.1" status: 204 len: 168 time: 0.1637075 May 20 04:36:12.638376 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-ecbd95e3-df5e-4064-bc33-b0138e0cc0e3 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:13.772663 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-34c1a1cc-5ca0-422d-b36d-32314e72315a req-14ca1685-8102-417c-bace-febf89fe4dcd tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:13.774093 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-34c1a1cc-5ca0-422d-b36d-32314e72315a req-14ca1685-8102-417c-bace-febf89fe4dcd tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=06b8328b-158c-4b35-9bc3-95b9eab9515c HTTP/1.1" status: 200 len: 945 time: 0.0854118 May 20 04:36:13.821993 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-34c1a1cc-5ca0-422d-b36d-32314e72315a req-95bafed8-10de-42f8-93fe-821e687dbfd8 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0396423 May 20 04:36:14.026209 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:36:14.975290 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:15.032021 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f4870576-1761-4c85-a252-cf693ffdda65 req-d4128ea6-e98c-4a54-8020-a100433f1a08 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:15.033633 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f4870576-1761-4c85-a252-cf693ffdda65 req-d4128ea6-e98c-4a54-8020-a100433f1a08 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=06b8328b-158c-4b35-9bc3-95b9eab9515c HTTP/1.1" status: 200 len: 945 time: 0.0859754 May 20 04:36:15.081855 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-f4870576-1761-4c85-a252-cf693ffdda65 req-d0042726-2aef-4d4f-8701-1e81f83594ce tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0424387 May 20 04:36:15.109844 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e6910d5d-2b0e-479a-9d86-ab24e22b1831 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:15.687713 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-84ec8bed-5b25-4571-82c4-a840ff092197 req-cb575d14-bfe2-436c-93f9-7948bac81bc9 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:15.688437 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-84ec8bed-5b25-4571-82c4-a840ff092197 req-cb575d14-bfe2-436c-93f9-7948bac81bc9 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Synchronizing usage tracker for tenant:1cc8a6f23e014d75bb434e4e526a41a2 on resource:port {{(pid=91644) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:36:15.708072 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-84ec8bed-5b25-4571-82c4-a840ff092197 req-cb575d14-bfe2-436c-93f9-7948bac81bc9 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Unset dirty status for tenant:1cc8a6f23e014d75bb434e4e526a41a2 on resource:port {{(pid=91644) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:36:15.709169 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-84ec8bed-5b25-4571-82c4-a840ff092197 req-cb575d14-bfe2-436c-93f9-7948bac81bc9 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.74 "GET /v2.0/ports?device_id=06b8328b-158c-4b35-9bc3-95b9eab9515c HTTP/1.1" status: 200 len: 945 time: 0.1143749 May 20 04:36:15.851011 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [req-84ec8bed-5b25-4571-82c4-a840ff092197 req-29261de6-a73b-4ccd-a3d4-89e86ddeeb3b tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Deleting port 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:36:15.851393 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-84ec8bed-5b25-4571-82c4-a840ff092197 req-29261de6-a73b-4ccd-a3d4-89e86ddeeb3b tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:15.984900 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-84ec8bed-5b25-4571-82c4-a840ff092197 req-29261de6-a73b-4ccd-a3d4-89e86ddeeb3b tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] For port 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d,segment=NetworkSegment(76847180-0644-4d57-b40f-7f63a056f8ec),segment_id=76847180-0644-4d57-b40f-7f63a056f8ec)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:36:15.985145 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-84ec8bed-5b25-4571-82c4-a840ff092197 req-29261de6-a73b-4ccd-a3d4-89e86ddeeb3b tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:16.108871 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [req-84ec8bed-5b25-4571-82c4-a840ff092197 req-29261de6-a73b-4ccd-a3d4-89e86ddeeb3b tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Calling delete_port for 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d owned by compute:nova {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:36:16.225304 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-84ec8bed-5b25-4571-82c4-a840ff092197 req-29261de6-a73b-4ccd-a3d4-89e86ddeeb3b tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:16.232087 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-45d169a7-ea3d-4dea-ad77-c50490b8ae7d req-aeaf37fb-c026-4993-be8a-f2fea9737829 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=06b8328b-158c-4b35-9bc3-95b9eab9515c HTTP/1.1" status: 200 len: 210 time: 0.0445154 May 20 04:36:16.235649 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-3c3bda92-1df2-4311-b8b9-b8e16e291e88 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:36:16.239674 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:36:16.289006 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:16.303127 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-29261de6-a73b-4ccd-a3d4-89e86ddeeb3b tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Pushing event deleted for resources: {'Port': ['ID=7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:36:16.313547 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-29261de6-a73b-4ccd-a3d4-89e86ddeeb3b tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:36:16.351909 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-84ec8bed-5b25-4571-82c4-a840ff092197 req-29261de6-a73b-4ccd-a3d4-89e86ddeeb3b tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] No DVR routers for this DVR port 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d on host hv-cinder-78240 {{(pid=91644) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:36:16.354622 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-84ec8bed-5b25-4571-82c4-a840ff092197 req-29261de6-a73b-4ccd-a3d4-89e86ddeeb3b tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:16.357153 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '06b8328b-158c-4b35-9bc3-95b9eab9515c', 'name': 'network-vif-deleted', 'tag': '7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:36:16.379071 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-84ec8bed-5b25-4571-82c4-a840ff092197 req-29261de6-a73b-4ccd-a3d4-89e86ddeeb3b tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.74 "DELETE /v2.0/ports/7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d HTTP/1.1" status: 204 len: 173 time: 0.6630540 May 20 04:36:16.405652 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '06b8328b-158c-4b35-9bc3-95b9eab9515c', 'name': 'network-vif-deleted', 'tag': '7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d', 'status': 'completed', 'code': 200} May 20 04:36:17.894984 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:20.038720 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:36:20.077715 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:36:20.078062 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device 7d7e5e0f-5a0e-40cc-8d70-d4b9702e3e7d not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:36:28.006168 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-7ce2d189-1708-41e4-b9d8-2b58c25499f5 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:28.007515 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-7ce2d189-1708-41e4-b9d8-2b58c25499f5 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/networks?id=bb372f0c-268d-4c71-bb09-cab162d5731a HTTP/1.1" status: 200 len: 814 time: 0.1626530 May 20 04:36:28.022687 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-1283b941-d913-494d-9d17-1b2f51334ac5 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/quotas/1cc8a6f23e014d75bb434e4e526a41a2 HTTP/1.1" status: 200 len: 375 time: 0.0096543 May 20 04:36:28.069019 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-13da68c5-8353-4c9e-ab6c-5374d08e5046 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?tenant_id=1cc8a6f23e014d75bb434e4e526a41a2&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0413761 May 20 04:36:28.179584 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-f127f731-dcf5-480c-a9a7-45aa57dac449 service neutron] http://10.222.0.39:9696/v2.0/extensions returned with HTTP 200 {{(pid=91643) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 20 04:36:28.180522 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-f127f731-dcf5-480c-a9a7-45aa57dac449 service neutron] 10.222.0.39 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.1050384 May 20 04:36:28.402140 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-70203114-af8f-4841-a6a3-71f626e865b5 service neutron] 10.222.0.39 "GET /v2.0/networks/bb372f0c-268d-4c71-bb09-cab162d5731a?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2155392 May 20 04:36:28.532719 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-09bdbe29-d3a8-4782-af9f-67f3d133fb58 service neutron] 10.222.0.39 "GET /v2.0/networks/bb372f0c-268d-4c71-bb09-cab162d5731a?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1246133 May 20 04:36:28.712062 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-318eccb9-df56-47c5-992a-c0253fe34f9b req-7e003889-fd42-41b4-8c5c-3204c8594595 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=90c03466-0a79-453f-8c0f-a72a583b5efa HTTP/1.1" status: 200 len: 210 time: 0.0455184 May 20 04:36:29.778124 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-1d868861-56e8-419b-bff5-ed768e108e7e tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:29.779580 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-1d868861-56e8-419b-bff5-ed768e108e7e tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.74 "GET /v2.0/networks?id=bb372f0c-268d-4c71-bb09-cab162d5731a HTTP/1.1" status: 200 len: 814 time: 0.1498678 May 20 04:36:29.791701 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-f47e2eda-d795-4a32-8312-64b9b7ac7aa4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Request body: {'port': {'device_id': '90c03466-0a79-453f-8c0f-a72a583b5efa', 'network_id': 'bb372f0c-268d-4c71-bb09-cab162d5731a', 'admin_state_up': True, 'tenant_id': '1cc8a6f23e014d75bb434e4e526a41a2'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:36:29.864086 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-29e7cbd8-a425-4c8a-b7ba-de06eaaa20e7 req-2e2d19da-16d7-4aca-8bdc-bcb2c470e275 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=90c03466-0a79-453f-8c0f-a72a583b5efa HTTP/1.1" status: 200 len: 210 time: 0.0428874 May 20 04:36:30.022333 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-f47e2eda-d795-4a32-8312-64b9b7ac7aa4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 20 04:36:30.026419 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-f47e2eda-d795-4a32-8312-64b9b7ac7aa4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Usage tracker for resource:port and tenant:1cc8a6f23e014d75bb434e4e526a41a2 is out of sync, need to count used quota {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 20 04:36:30.029279 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-f47e2eda-d795-4a32-8312-64b9b7ac7aa4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Quota usage for port was recalculated. Used quota:0. {{(pid=91644) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 20 04:36:30.035056 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.quota.driver [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-f47e2eda-d795-4a32-8312-64b9b7ac7aa4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=91644) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 20 04:36:30.048091 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-f47e2eda-d795-4a32-8312-64b9b7ac7aa4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Made reservation on behalf of 1cc8a6f23e014d75bb434e4e526a41a2 for: {'port': 1} {{(pid=91644) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 20 04:36:30.051578 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-f47e2eda-d795-4a32-8312-64b9b7ac7aa4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:30.215195 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_common [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-f47e2eda-d795-4a32-8312-64b9b7ac7aa4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Allocated IP 172.20.0.6 (bb372f0c-268d-4c71-bb09-cab162d5731a/953afe7b-ba49-4dd0-b9aa-085c509e22c4/b1664007-3baa-48c9-9aa2-d210e6f1dee8) {{(pid=91644) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 20 04:36:30.417485 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-f47e2eda-d795-4a32-8312-64b9b7ac7aa4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Notify callbacks [] for port, precommit_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:30.450516 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-f47e2eda-d795-4a32-8312-64b9b7ac7aa4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8760688497399', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_create {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:30.456254 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-255cb570-51af-4e36-b0c2-fc8df5013014 tempest-TestMinimumBasicScenario-293247754 tempest-TestMinimumBasicScenario-293247754-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:36:30.535205 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-f47e2eda-d795-4a32-8312-64b9b7ac7aa4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Network bb372f0c-268d-4c71-bb09-cab162d5731a is already hosted by enough agents. {{(pid=91644) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 20 04:36:30.538237 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.scheduler.base_scheduler [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-f47e2eda-d795-4a32-8312-64b9b7ac7aa4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Resources bound (agent type, host, resource id): {{(pid=91644) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 20 04:36:30.542096 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-f47e2eda-d795-4a32-8312-64b9b7ac7aa4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Received port after_create {{(pid=91644) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 20 04:36:30.572400 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f47e2eda-d795-4a32-8312-64b9b7ac7aa4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Pushing event updated for resources: {'Port': ['ID=b1664007-3baa-48c9-9aa2-d210e6f1dee8,revision_number=1']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:36:30.578716 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-f47e2eda-d795-4a32-8312-64b9b7ac7aa4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.122s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:36:30.607891 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-f47e2eda-d795-4a32-8312-64b9b7ac7aa4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:30.608489 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-f47e2eda-d795-4a32-8312-64b9b7ac7aa4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:30.632563 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-f47e2eda-d795-4a32-8312-64b9b7ac7aa4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.74 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 0.8446529 May 20 04:36:30.644942 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.v2.base [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Request body: {'port': {'device_id': '90c03466-0a79-453f-8c0f-a72a583b5efa', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78240'}} {{(pid=91644) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 20 04:36:30.762906 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:31.058725 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b3181eb5-9748-4071-923b-54116235825a req-be9fbb44-5632-45cc-8aba-35d068799159 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:31.061436 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b3181eb5-9748-4071-923b-54116235825a req-be9fbb44-5632-45cc-8aba-35d068799159 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=90c03466-0a79-453f-8c0f-a72a583b5efa HTTP/1.1" status: 200 len: 931 time: 0.0875814 May 20 04:36:31.107047 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b3181eb5-9748-4071-923b-54116235825a req-18cbb0d0-6191-4f00-bd99-73a828fffd2e tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0410836 May 20 04:36:31.116072 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id b1664007-3baa-48c9-9aa2-d210e6f1dee8 {{(pid=91644) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:36:31.127956 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Notify callbacks [] for port, precommit_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:31.143180 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.provisioning_blocks [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Transition to ACTIVE for port object b1664007-3baa-48c9-9aa2-d210e6f1dee8 will not be triggered until provisioned by entity L2. {{(pid=91644) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 20 04:36:31.148580 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:31.152247 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-479708a4-3b98-48d5-9d75-cd50fd49f856 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:36:31.306533 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Pushing event updated for resources: {'Port': ['ID=b1664007-3baa-48c9-9aa2-d210e6f1dee8,revision_number=2']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:36:31.314287 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.162s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:36:31.397509 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Attempting to bind port b1664007-3baa-48c9-9aa2-d210e6f1dee8 on host hv-cinder-78240 for vnic_type normal with profile {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 20 04:36:31.397839 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Attempting to bind port b1664007-3baa-48c9-9aa2-d210e6f1dee8 by drivers openvswitch,hyperv on host hv-cinder-78240 at level 0 using segments [{'id': '76847180-0644-4d57-b40f-7f63a056f8ec', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 830, 'network_id': 'bb372f0c-268d-4c71-bb09-cab162d5731a'}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 20 04:36:31.398165 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Attempting to bind port b1664007-3baa-48c9-9aa2-d210e6f1dee8 on network bb372f0c-268d-4c71-bb09-cab162d5731a {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:36:31.402866 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Port b1664007-3baa-48c9-9aa2-d210e6f1dee8 on network bb372f0c-268d-4c71-bb09-cab162d5731a not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78240 {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 20 04:36:31.403180 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Attempting to bind port b1664007-3baa-48c9-9aa2-d210e6f1dee8 on network bb372f0c-268d-4c71-bb09-cab162d5731a {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 20 04:36:31.408481 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Checking agent: {'id': 'f793de9d-596f-4930-970b-f2b32ddc4689', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78240', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'started_at': datetime.datetime(2021, 5, 20, 4, 20, 6), 'heartbeat_timestamp': datetime.datetime(2021, 5, 20, 4, 36, 6), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 20 04:36:31.408812 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Checking segment: {'id': '76847180-0644-4d57-b40f-7f63a056f8ec', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 830, 'network_id': 'bb372f0c-268d-4c71-bb09-cab162d5731a'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=91644) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 20 04:36:31.409199 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Bound using segment: {'id': '76847180-0644-4d57-b40f-7f63a056f8ec', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 830, 'network_id': 'bb372f0c-268d-4c71-bb09-cab162d5731a'} {{(pid=91644) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 20 04:36:31.409643 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.managers [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Bound port: b1664007-3baa-48c9-9aa2-d210e6f1dee8, host: hv-cinder-78240, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '76847180-0644-4d57-b40f-7f63a056f8ec', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 830, 'network_id': 'bb372f0c-268d-4c71-bb09-cab162d5731a'}}] {{(pid=91644) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 20 04:36:31.409979 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:31.505998 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] For port b1664007-3baa-48c9-9aa2-d210e6f1dee8, host hv-cinder-78240, cleared binding levels {{(pid=91644) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 20 04:36:31.508950 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] For port b1664007-3baa-48c9-9aa2-d210e6f1dee8, host hv-cinder-78240, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b1664007-3baa-48c9-9aa2-d210e6f1dee8,segment=NetworkSegment(76847180-0644-4d57-b40f-7f63a056f8ec),segment_id=76847180-0644-4d57-b40f-7f63a056f8ec)] {{(pid=91644) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 20 04:36:31.516346 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-2001147'] for port, after_update {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:31.520759 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-d48c3948-7a6b-49c3-8a97-994eb8227138 tempest-TestVolumeBootPattern-1804082964 tempest-TestVolumeBootPattern-1804082964-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:36:31.574458 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:31.575063 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:31.576393 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-59b64498-b566-4355-8a95-140e796e8352 service neutron] 10.222.0.74 "PUT /v2.0/ports/b1664007-3baa-48c9-9aa2-d210e6f1dee8 HTTP/1.1" status: 200 len: 1093 time: 0.9354422 May 20 04:36:31.578911 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '90c03466-0a79-453f-8c0f-a72a583b5efa', 'tag': 'b1664007-3baa-48c9-9aa2-d210e6f1dee8'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:36:31.600319 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Pushing event updated for resources: {'Port': ['ID=b1664007-3baa-48c9-9aa2-d210e6f1dee8,revision_number=3']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:36:31.603499 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-59b64498-b566-4355-8a95-140e796e8352 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.083s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:36:31.647968 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '90c03466-0a79-453f-8c0f-a72a583b5efa', 'tag': 'b1664007-3baa-48c9-9aa2-d210e6f1dee8', 'status': 'completed', 'code': 200} May 20 04:36:31.692151 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-f64cc650-1b11-4b30-83e2-42cf1a9607d7 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:31.693655 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-f64cc650-1b11-4b30-83e2-42cf1a9607d7 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=1cc8a6f23e014d75bb434e4e526a41a2&device_id=90c03466-0a79-453f-8c0f-a72a583b5efa HTTP/1.1" status: 200 len: 1096 time: 0.0798175 May 20 04:36:31.761213 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-734905b4-34a7-4966-953e-80134cf81c16 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=b1664007-3baa-48c9-9aa2-d210e6f1dee8 HTTP/1.1" status: 200 len: 217 time: 0.0602322 May 20 04:36:31.916947 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-ed305fe6-b099-4b01-9919-9c3425c13d37 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:31.918305 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-ed305fe6-b099-4b01-9919-9c3425c13d37 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=953afe7b-ba49-4dd0-b9aa-085c509e22c4 HTTP/1.1" status: 200 len: 854 time: 0.1504149 May 20 04:36:31.965249 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-950e8d05-36da-41b4-8819-287050b597e4 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=bb372f0c-268d-4c71-bb09-cab162d5731a&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0399137 May 20 04:36:32.101759 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-112e3bc1-80af-426a-923f-e91c6c0c539c service neutron] 10.222.0.74 "GET /v2.0/networks/bb372f0c-268d-4c71-bb09-cab162d5731a?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1296253 May 20 04:36:32.235936 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-680eaf1a-cdf8-471b-b749-72ffed018de8 req-8f54bb56-f117-4e40-9461-2d75aff19559 service neutron] 10.222.0.74 "GET /v2.0/networks/bb372f0c-268d-4c71-bb09-cab162d5731a?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1267145 May 20 04:36:32.292443 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-413dc04c-f1ab-4a3a-81d4-9663ad991629 req-0b6f6572-8486-4eb0-9107-b4d1e80cef67 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:32.293909 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-413dc04c-f1ab-4a3a-81d4-9663ad991629 req-0b6f6572-8486-4eb0-9107-b4d1e80cef67 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=90c03466-0a79-453f-8c0f-a72a583b5efa HTTP/1.1" status: 200 len: 943 time: 0.0829074 May 20 04:36:32.339082 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-413dc04c-f1ab-4a3a-81d4-9663ad991629 req-14b573cd-ad9d-4914-94bf-d52bfea524fd tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0390556 May 20 04:36:32.360962 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7bdfdbd-fd31-4f60-bf1f-ec995ad7ea99 req-0cdfcd56-a75e-4efc-b2f3-1704fa3a1df0 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:32.362435 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-a7bdfdbd-fd31-4f60-bf1f-ec995ad7ea99 req-0cdfcd56-a75e-4efc-b2f3-1704fa3a1df0 service neutron] 10.222.0.74 "GET /v2.0/ports?tenant_id=1cc8a6f23e014d75bb434e4e526a41a2&device_id=90c03466-0a79-453f-8c0f-a72a583b5efa HTTP/1.1" status: 200 len: 1096 time: 0.0758052 May 20 04:36:32.502208 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7bdfdbd-fd31-4f60-bf1f-ec995ad7ea99 req-df9bebd5-c112-4e51-9c5c-a3dbf6a5860a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:32.504404 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-a7bdfdbd-fd31-4f60-bf1f-ec995ad7ea99 req-df9bebd5-c112-4e51-9c5c-a3dbf6a5860a service neutron] 10.222.0.74 "GET /v2.0/networks?id=bb372f0c-268d-4c71-bb09-cab162d5731a HTTP/1.1" status: 200 len: 915 time: 0.1350062 May 20 04:36:32.581563 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-a7bdfdbd-fd31-4f60-bf1f-ec995ad7ea99 req-78c09cda-ff07-4136-9e69-82f99859e291 service neutron] 10.222.0.74 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.6&port_id=b1664007-3baa-48c9-9aa2-d210e6f1dee8 HTTP/1.1" status: 200 len: 217 time: 0.0700607 May 20 04:36:32.649360 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7bdfdbd-fd31-4f60-bf1f-ec995ad7ea99 req-32cb947c-729b-4b38-8e42-a1d9d083ff72 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:32.651572 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-a7bdfdbd-fd31-4f60-bf1f-ec995ad7ea99 req-32cb947c-729b-4b38-8e42-a1d9d083ff72 service neutron] 10.222.0.74 "GET /v2.0/subnets?id=953afe7b-ba49-4dd0-b9aa-085c509e22c4 HTTP/1.1" status: 200 len: 854 time: 0.0631850 May 20 04:36:32.705433 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-a7bdfdbd-fd31-4f60-bf1f-ec995ad7ea99 req-2fecb657-228b-4a72-b4d6-39087d1d0281 service neutron] 10.222.0.74 "GET /v2.0/ports?network_id=bb372f0c-268d-4c71-bb09-cab162d5731a&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0462763 May 20 04:36:32.838677 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-a7bdfdbd-fd31-4f60-bf1f-ec995ad7ea99 req-38a3384f-3694-4978-9770-eee829248b64 service neutron] 10.222.0.74 "GET /v2.0/networks/bb372f0c-268d-4c71-bb09-cab162d5731a?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1262262 May 20 04:36:32.971493 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-a7bdfdbd-fd31-4f60-bf1f-ec995ad7ea99 req-d688091c-d2e3-48bd-85aa-9d2d74cf29d7 service neutron] 10.222.0.74 "GET /v2.0/networks/bb372f0c-268d-4c71-bb09-cab162d5731a?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1261768 May 20 04:36:33.539157 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d57f9e7f-c026-4045-8094-27952c0d6b8f req-4f49bf5c-6f8b-4f42-a5a2-2c1682b05c58 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:33.540605 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d57f9e7f-c026-4045-8094-27952c0d6b8f req-4f49bf5c-6f8b-4f42-a5a2-2c1682b05c58 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=90c03466-0a79-453f-8c0f-a72a583b5efa HTTP/1.1" status: 200 len: 943 time: 0.0874314 May 20 04:36:33.585727 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-d57f9e7f-c026-4045-8094-27952c0d6b8f req-2fd17607-f439-4d3e-aaca-8bde3645ec1a tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0393827 May 20 04:36:34.789493 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e6514013-24dc-4535-aba5-b8bdfc44db50 req-7f8f9dc8-cfa6-4b52-ae9c-c62482a7726d tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:34.790921 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e6514013-24dc-4535-aba5-b8bdfc44db50 req-7f8f9dc8-cfa6-4b52-ae9c-c62482a7726d tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=90c03466-0a79-453f-8c0f-a72a583b5efa HTTP/1.1" status: 200 len: 943 time: 0.0893431 May 20 04:36:34.840989 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-e6514013-24dc-4535-aba5-b8bdfc44db50 req-63e7104b-2970-4db3-9493-ed43b0cf02f1 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0415814 May 20 04:36:35.732963 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:36.037473 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1760f2bc-efd3-4f2a-9aeb-c7fb7d59fb2f req-2f0a1853-c171-46fb-a15f-0e2a34ab0ff5 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:36.038811 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1760f2bc-efd3-4f2a-9aeb-c7fb7d59fb2f req-2f0a1853-c171-46fb-a15f-0e2a34ab0ff5 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=90c03466-0a79-453f-8c0f-a72a583b5efa HTTP/1.1" status: 200 len: 943 time: 0.0835454 May 20 04:36:36.063599 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b1664007-3baa-48c9-9aa2-d210e6f1dee8 details requested by agent hyperv_hv-cinder-78240 with host hv-cinder-78240 {{(pid=91645) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 20 04:36:36.084736 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1760f2bc-efd3-4f2a-9aeb-c7fb7d59fb2f req-274130a2-8e9c-4eca-a003-d190f18fd089 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0401068 May 20 04:36:36.201769 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b1664007-3baa-48c9-9aa2-d210e6f1dee8, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b1664007-3baa-48c9-9aa2-d210e6f1dee8,segment=NetworkSegment(76847180-0644-4d57-b40f-7f63a056f8ec),segment_id=76847180-0644-4d57-b40f-7f63a056f8ec)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:36:36.214810 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Returning: {'device': 'b1664007-3baa-48c9-9aa2-d210e6f1dee8', 'network_id': 'bb372f0c-268d-4c71-bb09-cab162d5731a', 'port_id': 'b1664007-3baa-48c9-9aa2-d210e6f1dee8', 'mac_address': 'fa:16:3e:f9:78:92', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 830, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '953afe7b-ba49-4dd0-b9aa-085c509e22c4', 'ip_address': '172.20.0.6'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=91645) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 20 04:36:36.273923 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:36.290228 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id b1664007-3baa-48c9-9aa2-d210e6f1dee8 {{(pid=91645) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 20 04:36:36.305523 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b1664007-3baa-48c9-9aa2-d210e6f1dee8, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b1664007-3baa-48c9-9aa2-d210e6f1dee8,segment=NetworkSegment(76847180-0644-4d57-b40f-7f63a056f8ec),segment_id=76847180-0644-4d57-b40f-7f63a056f8ec)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:36:36.312020 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:36.313700 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:36:36.378016 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=b1664007-3baa-48c9-9aa2-d210e6f1dee8,revision_number=4']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:36:36.385287 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:36.385635 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:36.385969 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:36.386300 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:36.386628 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:36.386953 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:36.387307 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:36.387624 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:36.387950 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:36.388246 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:36.388561 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:36.388878 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91645) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:36:36.392669 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.079s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:36:36.640018 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] get_ports_and_sgs() called for port_ids ['b1664007-3baa-48c9-9aa2-d210e6f1dee8'] {{(pid=91645) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 20 04:36:37.289980 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e88466e-6f7c-442c-a294-6ebb82b60017 req-a4f1542b-82fa-41ca-852d-3e3632c0c83e tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:37.291496 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8e88466e-6f7c-442c-a294-6ebb82b60017 req-a4f1542b-82fa-41ca-852d-3e3632c0c83e tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=90c03466-0a79-453f-8c0f-a72a583b5efa HTTP/1.1" status: 200 len: 944 time: 0.0990467 May 20 04:36:37.336984 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-8e88466e-6f7c-442c-a294-6ebb82b60017 req-1a5f2484-bf70-4578-8949-27a027dda9ab tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0395904 May 20 04:36:38.080545 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b1664007-3baa-48c9-9aa2-d210e6f1dee8 up at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 20 04:36:38.160051 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning for port b1664007-3baa-48c9-9aa2-d210e6f1dee8 completed by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 20 04:36:38.164736 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.db.provisioning_blocks [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Provisioning complete for port b1664007-3baa-48c9-9aa2-d210e6f1dee8 triggered by entity L2. {{(pid=91645) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 20 04:36:38.165113 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-7783085'] for port, provisioning_complete {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:38.281093 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-585843'] for port, before_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:38.321264 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '90c03466-0a79-453f-8c0f-a72a583b5efa', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b1664007-3baa-48c9-9aa2-d210e6f1dee8'}] {{(pid=91645) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:36:38.331650 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.db [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] For port b1664007-3baa-48c9-9aa2-d210e6f1dee8, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b1664007-3baa-48c9-9aa2-d210e6f1dee8,segment=NetworkSegment(76847180-0644-4d57-b40f-7f63a056f8ec),segment_id=76847180-0644-4d57-b40f-7f63a056f8ec)] {{(pid=91645) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:36:38.337143 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8760688930991', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8760688499725', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8760688499770'] for port, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:38.340676 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:36:38.372786 dv-cinder-782405-9-iscsi neutron-server[91645]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '90c03466-0a79-453f-8c0f-a72a583b5efa', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b1664007-3baa-48c9-9aa2-d210e6f1dee8', 'code': 200} May 20 04:36:38.414665 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Pushing event updated for resources: {'Port': ['ID=b1664007-3baa-48c9-9aa2-d210e6f1dee8,revision_number=5']} {{(pid=91645) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:36:38.417834 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG oslo_concurrency.lockutils [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=91645) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:36:38.530172 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-16d43f9d-82ef-469e-b520-135f7ccc168c req-155f5b7a-ee0e-45ab-9d82-1aa661782dc6 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:38.531643 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-16d43f9d-82ef-469e-b520-135f7ccc168c req-155f5b7a-ee0e-45ab-9d82-1aa661782dc6 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=90c03466-0a79-453f-8c0f-a72a583b5efa HTTP/1.1" status: 200 len: 945 time: 0.0816476 May 20 04:36:38.576505 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-16d43f9d-82ef-469e-b520-135f7ccc168c req-55178314-209b-457c-8628-694d3fabf760 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0389037 May 20 04:36:38.597852 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7540a160-4619-4586-8616-7a4026cdd7de req-c7926e5f-ca2a-4d5d-a026-6a39829c1499 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:38.599173 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7540a160-4619-4586-8616-7a4026cdd7de req-c7926e5f-ca2a-4d5d-a026-6a39829c1499 service neutron] 10.222.0.74 "GET /v2.0/ports?device_id=90c03466-0a79-453f-8c0f-a72a583b5efa HTTP/1.1" status: 200 len: 1098 time: 0.0744693 May 20 04:36:38.638832 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-7540a160-4619-4586-8616-7a4026cdd7de req-0f20bcfc-66dc-4188-b245-6fad88693b6e service neutron] 10.222.0.74 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0333292 May 20 04:36:39.780343 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-243a6a70-b30f-4f43-85d9-ad78772ea127 req-4e3158be-6928-4c15-8a45-8c5d8eaa90d8 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:39.781844 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-243a6a70-b30f-4f43-85d9-ad78772ea127 req-4e3158be-6928-4c15-8a45-8c5d8eaa90d8 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=90c03466-0a79-453f-8c0f-a72a583b5efa HTTP/1.1" status: 200 len: 945 time: 0.0872128 May 20 04:36:39.829166 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-243a6a70-b30f-4f43-85d9-ad78772ea127 req-adcdc787-bce5-43c4-9bfb-086bf6f139b2 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0412221 May 20 04:36:41.036703 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b578f63b-a152-4d27-8275-4c0ff08b7618 req-ea4e312c-ce9e-4704-8119-424501ded6ae tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:36:41.038112 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b578f63b-a152-4d27-8275-4c0ff08b7618 req-ea4e312c-ce9e-4704-8119-424501ded6ae tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=90c03466-0a79-453f-8c0f-a72a583b5efa HTTP/1.1" status: 200 len: 945 time: 0.0846143 May 20 04:36:41.086858 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-b578f63b-a152-4d27-8275-4c0ff08b7618 req-93167dca-c2e2-4e79-8874-c7d88494b28c tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0430079 May 20 04:36:42.638835 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-7c2e24a7-f659-4966-bece-1ee6535050b0 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:45.019782 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:45.116972 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-46e0278b-7a64-46bb-ac5a-3494ba3ff290 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:46.256811 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:47.893185 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:36:51.030303 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:37:05.719705 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:06.137582 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1b9ca800-0e0d-4c05-aa09-748a6e0ec4f9 req-8248ead6-2332-47dc-b8c5-2642714a2600 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:37:06.139005 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1b9ca800-0e0d-4c05-aa09-748a6e0ec4f9 req-8248ead6-2332-47dc-b8c5-2642714a2600 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=90c03466-0a79-453f-8c0f-a72a583b5efa HTTP/1.1" status: 200 len: 945 time: 0.0877352 May 20 04:37:06.185069 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-1b9ca800-0e0d-4c05-aa09-748a6e0ec4f9 req-1f9ae18b-b211-4227-9e5f-a61325950eaa tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0387368 May 20 04:37:07.449170 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-56bb0bb1-36b7-40b6-8cc5-c0835d153e04 req-a2c9831b-c232-49e8-9d77-0fb70f7b8e5c tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:37:07.450566 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-56bb0bb1-36b7-40b6-8cc5-c0835d153e04 req-a2c9831b-c232-49e8-9d77-0fb70f7b8e5c tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=90c03466-0a79-453f-8c0f-a72a583b5efa HTTP/1.1" status: 200 len: 945 time: 0.1420844 May 20 04:37:07.496926 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-56bb0bb1-36b7-40b6-8cc5-c0835d153e04 req-c36f52b6-bc82-4f8d-bd65-aa64c7d8ffdd tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0405264 May 20 04:37:08.030117 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-93d604bd-a4cd-48f4-85c2-0e2ad5d4dfa4 req-f3c5be06-da45-4119-a2e3-9034b3cd3ca4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91644) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:37:08.030749 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-93d604bd-a4cd-48f4-85c2-0e2ad5d4dfa4 req-f3c5be06-da45-4119-a2e3-9034b3cd3ca4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Synchronizing usage tracker for tenant:1cc8a6f23e014d75bb434e4e526a41a2 on resource:port {{(pid=91644) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 20 04:37:08.050778 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.quota.resource [req-93d604bd-a4cd-48f4-85c2-0e2ad5d4dfa4 req-f3c5be06-da45-4119-a2e3-9034b3cd3ca4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Unset dirty status for tenant:1cc8a6f23e014d75bb434e4e526a41a2 on resource:port {{(pid=91644) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 20 04:37:08.051990 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-93d604bd-a4cd-48f4-85c2-0e2ad5d4dfa4 req-f3c5be06-da45-4119-a2e3-9034b3cd3ca4 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.74 "GET /v2.0/ports?device_id=90c03466-0a79-453f-8c0f-a72a583b5efa HTTP/1.1" status: 200 len: 945 time: 0.1183107 May 20 04:37:08.193873 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [req-93d604bd-a4cd-48f4-85c2-0e2ad5d4dfa4 req-ab6fda2c-0530-4849-a171-373d54eb197a tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Deleting port b1664007-3baa-48c9-9aa2-d210e6f1dee8 {{(pid=91644) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 20 04:37:08.194108 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-93d604bd-a4cd-48f4-85c2-0e2ad5d4dfa4 req-ab6fda2c-0530-4849-a171-373d54eb197a tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8760688928615'] for port, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:08.326464 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.db [req-93d604bd-a4cd-48f4-85c2-0e2ad5d4dfa4 req-ab6fda2c-0530-4849-a171-373d54eb197a tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] For port b1664007-3baa-48c9-9aa2-d210e6f1dee8, host hv-cinder-78240, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78240',level=0,port_id=b1664007-3baa-48c9-9aa2-d210e6f1dee8,segment=NetworkSegment(76847180-0644-4d57-b40f-7f63a056f8ec),segment_id=76847180-0644-4d57-b40f-7f63a056f8ec)] {{(pid=91644) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 20 04:37:08.326813 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-93d604bd-a4cd-48f4-85c2-0e2ad5d4dfa4 req-ab6fda2c-0530-4849-a171-373d54eb197a tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8001935'] for port, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:08.536449 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.plugins.ml2.plugin [req-93d604bd-a4cd-48f4-85c2-0e2ad5d4dfa4 req-ab6fda2c-0530-4849-a171-373d54eb197a tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Calling delete_port for b1664007-3baa-48c9-9aa2-d210e6f1dee8 owned by compute:nova {{(pid=91644) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 20 04:37:08.652475 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-93d604bd-a4cd-48f4-85c2-0e2ad5d4dfa4 req-ab6fda2c-0530-4849-a171-373d54eb197a tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-587656', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8190021', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8760688931000', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8760688499716', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-2001122'] for port, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:08.659326 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-a99a3227-7799-4f92-980a-32ef1d462ffd admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:37:08.683873 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a61720ec-5af4-4274-878e-35fbb302b611 req-b525441f-5947-4167-bde1-5c90e332b9c0 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=91643) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 20 04:37:08.685361 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a61720ec-5af4-4274-878e-35fbb302b611 req-b525441f-5947-4167-bde1-5c90e332b9c0 tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/ports?device_id=90c03466-0a79-453f-8c0f-a72a583b5efa HTTP/1.1" status: 200 len: 945 time: 0.0813370 May 20 04:37:08.728789 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ab6fda2c-0530-4849-a171-373d54eb197a tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Pushing event deleted for resources: {'Port': ['ID=b1664007-3baa-48c9-9aa2-d210e6f1dee8,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:37:08.731238 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-ab6fda2c-0530-4849-a171-373d54eb197a tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.072s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:37:08.731751 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [req-a61720ec-5af4-4274-878e-35fbb302b611 req-01e09267-1837-48dd-809e-d638053262dc tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.39 "GET /v2.0/security-groups?id=6f0d273f-0b64-4321-8661-9765ae71fbcb&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0403240 May 20 04:37:08.779249 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.l3_dvrscheduler_db [req-93d604bd-a4cd-48f4-85c2-0e2ad5d4dfa4 req-ab6fda2c-0530-4849-a171-373d54eb197a tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] No DVR routers for this DVR port b1664007-3baa-48c9-9aa2-d210e6f1dee8 on host hv-cinder-78240 {{(pid=91644) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 20 04:37:08.782481 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [req-93d604bd-a4cd-48f4-85c2-0e2ad5d4dfa4 req-ab6fda2c-0530-4849-a171-373d54eb197a tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-13269'] for port, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:08.787268 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '90c03466-0a79-453f-8c0f-a72a583b5efa', 'name': 'network-vif-deleted', 'tag': 'b1664007-3baa-48c9-9aa2-d210e6f1dee8'}] {{(pid=91644) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 20 04:37:08.804338 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [req-93d604bd-a4cd-48f4-85c2-0e2ad5d4dfa4 req-ab6fda2c-0530-4849-a171-373d54eb197a tempest-VolumesSnapshotTestJSON-1223942043 tempest-VolumesSnapshotTestJSON-1223942043-project] 10.222.0.74 "DELETE /v2.0/ports/b1664007-3baa-48c9-9aa2-d210e6f1dee8 HTTP/1.1" status: 204 len: 173 time: 0.7461159 May 20 04:37:08.837950 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '90c03466-0a79-453f-8c0f-a72a583b5efa', 'name': 'network-vif-deleted', 'tag': 'b1664007-3baa-48c9-9aa2-d210e6f1dee8', 'status': 'completed', 'code': 200} May 20 04:37:10.187876 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b1664007-3baa-48c9-9aa2-d210e6f1dee8 no longer exists at agent hyperv_hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 20 04:37:10.221684 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.plugin [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] No Port match for: b1664007-3baa-48c9-9aa2-d210e6f1dee8 {{(pid=91645) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 20 04:37:10.222030 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron.plugins.ml2.rpc [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Device b1664007-3baa-48c9-9aa2-d210e6f1dee8 not bound to the agent host hv-cinder-78240 {{(pid=91645) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 20 04:37:12.639152 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-aab24963-68db-400f-a4d8-59679ade5b87 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:14.999328 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:15.113696 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e16037f0-3ef1-4e40-9ec4-6a952a381bd6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:16.255898 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:17.892139 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:28.030565 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:37:35.719530 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:37.808173 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 46090) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:37:37.878390 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.db.db_base_plugin_v2 [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] Deleting subnet 953afe7b-ba49-4dd0-b9aa-085c509e22c4 {{(pid=91644) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 20 04:37:37.881705 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:37.933225 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-853257'] for subnet, precommit_delete_associations {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:37.998611 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-7786687'] for subnet, precommit_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:38.250802 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-7786632', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-8231094', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-7924868', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-1988925', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-1810229'] for subnet, after_delete {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:38.254715 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-6350424d-0718-4e5b-948b-ded8293f840f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:37:38.288163 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=953afe7b-ba49-4dd0-b9aa-085c509e22c4,revision_number=None']} {{(pid=91644) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:37:38.292727 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron_lib.callbacks.manager [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] Notify callbacks [] for subnet, before_response {{(pid=91644) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:38.302012 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.302382 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.302742 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.303116 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.303495 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.303846 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.304196 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.304543 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.304891 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.305237 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.305562 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.305903 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91644) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.308846 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG oslo_concurrency.lockutils [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=91644) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:37:38.310487 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-7e43bd84-aa43-4578-9904-54ed97c35420 admin admin] 10.222.0.39 "DELETE /v2.0/subnets/953afe7b-ba49-4dd0-b9aa-085c509e22c4 HTTP/1.1" status: 204 len: 168 time: 0.5010045 May 20 04:37:38.314425 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 46094) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:37:38.432376 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-593607'] for network, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:38.640395 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-7789108', 'neutron.services.segments.db._delete_segments_for_network-8760688958082'] for network, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:38.656232 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['bb372f0c-268d-4c71-bb09-cab162d5731a']}} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:37:38.668813 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '76847180-0644-4d57-b40f-7f63a056f8ec') {'for_net_delete': True} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:37:38.669104 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.services.segments.db [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '76847180-0644-4d57-b40f-7f63a056f8ec') {} {{(pid=91643) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 20 04:37:38.680873 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8760688968331', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-1472741'] for segment, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:38.698516 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284'] for segment, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:38.704925 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Releasing vlan 830 on physical network physnet1 to pool {{(pid=91643) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 20 04:37:38.705296 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-7778284', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-2070625'] for segment, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: Traceback (most recent call last): May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: timer() May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: cb(*args, **kw) May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: return func(*args, **kwargs) May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: self._notify() May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: self.callback(batched_events) May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: event.method(event) May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: aggregate_id = self._get_aggregate_id(event.segment_id) May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: aggregate_uuid = self.p_client.list_aggregates( May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: return f(self, *a, **k) May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: return self._get(url).json() May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: return self._client.get(url, endpoint_filter=self._ks_filter, May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: return self.request(url, 'GET', **kwargs) May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: base_url = self.get_endpoint(auth, allow=allow, May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: auth = self._auth_required(auth, 'determine endpoint URL') May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 20 04:37:38.718022 dv-cinder-782405-9-iscsi neutron-server[91643]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 20 04:37:38.792574 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-7789117', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-7895949', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-7924915', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-8053321'] for network, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:38.795953 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:37:38.812104 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-7924886'] for network, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:38.822360 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Pushing event deleted for resources: {'Network': ['ID=bb372f0c-268d-4c71-bb09-cab162d5731a,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:37:38.831385 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.831771 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.832142 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.832492 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.832855 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.833229 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.833570 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.833925 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.834279 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.834632 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.834985 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.835365 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-782405-9-iscsi {{(pid=91643) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 20 04:37:38.838901 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] 10.222.0.39 "DELETE /v2.0/networks/bb372f0c-268d-4c71-bb09-cab162d5731a HTTP/1.1" status: 204 len: 168 time: 0.5234947 May 20 04:37:38.839846 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-981d8994-a6be-4913-94b9-12c1b5872083 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:37:39.039958 dv-cinder-782405-9-iscsi neutron-server[91644]: DEBUG neutron.wsgi [-] (91644) accepted ('10.222.0.39', 46100) {{(pid=91644) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:37:39.090692 dv-cinder-782405-9-iscsi neutron-server[91644]: INFO neutron.wsgi [None req-8b59e3a3-eb65-4fa2-b834-b65675f07f5f admin admin] 10.222.0.39 "GET /v2.0/security-groups?tenant_id=1cc8a6f23e014d75bb434e4e526a41a2&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0495250 May 20 04:37:39.093732 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.wsgi [-] (91643) accepted ('10.222.0.39', 46104) {{(pid=91643) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 20 04:37:39.163412 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-772c06c4-de26-4b69-9f10-73827c2e7511 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:39.189601 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-772c06c4-de26-4b69-9f10-73827c2e7511 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:39.246466 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-772c06c4-de26-4b69-9f10-73827c2e7511 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-7895935'] for security_group, after_delete {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:39.251132 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-953c9854-4b78-4b59-b1b1-176cccc2c2b4 tempest-AttachVolumeTestJSON-1534523648 tempest-AttachVolumeTestJSON-1534523648-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 20 04:37:39.263324 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-772c06c4-de26-4b69-9f10-73827c2e7511 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=6f0d273f-0b64-4321-8661-9765ae71fbcb,revision_number=None']} {{(pid=91643) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 20 04:37:39.266114 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG oslo_concurrency.lockutils [None req-772c06c4-de26-4b69-9f10-73827c2e7511 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=91643) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 20 04:37:39.274981 dv-cinder-782405-9-iscsi neutron-server[91643]: DEBUG neutron_lib.callbacks.manager [None req-772c06c4-de26-4b69-9f10-73827c2e7511 admin admin] Notify callbacks [] for security_group, before_response {{(pid=91643) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:39.276289 dv-cinder-782405-9-iscsi neutron-server[91643]: INFO neutron.wsgi [None req-772c06c4-de26-4b69-9f10-73827c2e7511 admin admin] 10.222.0.39 "DELETE /v2.0/security-groups/6f0d273f-0b64-4321-8661-9765ae71fbcb HTTP/1.1" status: 204 len: 168 time: 0.1816368 May 20 04:37:42.639733 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-be392270-435d-4bd9-9e29-f4c77a4f6481 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:44.989784 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:45.109797 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e9f64302-358b-4fae-ac5c-e2d7597ef8f7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:46.262062 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:37:47.899678 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:38:05.034597 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:38:05.735122 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:38:12.635264 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-7b31c426-fd25-4714-9e88-57243e2de5c4 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:38:14.993512 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:38:15.109293 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e4756ee9-d07f-45a5-93af-4bbfdff97e44 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:38:16.271692 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:38:17.897566 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:38:35.742317 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:38:42.036983 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:38:42.630738 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-7692b254-f9ce-48ba-b1f7-5024a8c498b2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:38:45.002337 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:38:45.109839 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-6cd55f5f-5111-4419-8416-b4fdad0e4af5 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:38:46.274034 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:38:47.901539 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:39:05.772371 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:39:12.636017 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-cfd20e90-a056-4e22-9d20-84cdd42a70a0 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:39:15.002412 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:39:15.109325 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-a3af09bd-676e-4054-896c-77ec41d5a7d6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:39:16.276779 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:39:17.906040 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:39:19.041869 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:39:35.783423 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:39:42.638792 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-422d2f44-4b76-444f-a86e-706fce7dfa19 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:39:45.001308 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:39:45.109932 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-e83b5c67-9bd5-40b2-bf82-d9808b9f7d6e None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:39:46.278841 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:39:47.908624 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:39:56.047415 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:40:05.781509 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:40:12.639619 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-6fe26166-233a-4402-abe1-334795a8ce23 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:40:14.989280 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:40:15.107909 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-41eccede-43d7-4bd4-9bb7-29b3a13847ab None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:40:16.283269 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:40:17.918096 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:40:33.047062 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:40:35.799641 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:40:42.639974 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-01997258-f803-410b-99bd-225fb0fbc7c8 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:40:44.998549 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:40:45.109656 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-73ba80b6-b1f1-4611-8563-788ebef41220 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:40:46.291025 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:40:47.921538 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:41:05.796678 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:41:10.051682 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:41:12.641685 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-725783fa-7678-4858-8512-ac992edb3170 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:41:14.992516 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:41:15.105370 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0970623e-1f03-45d2-ad75-c7e209f2c197 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:41:16.295409 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:41:17.926100 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:41:35.798949 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:41:42.643034 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-28a35f99-a9c8-4f84-9b25-1cdf9ca4c2aa None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:41:45.000887 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:41:45.112416 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-3fefb9c4-5b7f-47f5-9764-7067abf80097 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:41:46.300373 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:41:47.061554 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:41:47.923287 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:42:05.814644 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:42:12.644877 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-dcce1ce5-fa6c-4306-b9fb-ba61b8af50eb None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:42:15.001106 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:42:15.107452 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-d9b11dbe-7f74-4101-8993-a61abaa2f75b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:42:16.301322 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:42:17.930768 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:42:24.055078 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}} May 20 04:42:35.839978 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-c3acd5c9-9389-4efc-bd38-258075bf36b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:42:42.646287 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-f9baad0e-fe5f-4d87-bc61-96f4c8b4a99d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:42:45.000264 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-198357cd-aaad-4397-a100-ced747a19175 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:42:45.116425 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-3557ab5d-9b05-4932-9d23-7e69745e449b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:42:46.305610 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-0a7cfb25-1644-4171-a92f-de73877aa22a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:42:47.935485 dv-cinder-782405-9-iscsi neutron-server[91645]: DEBUG neutron_lib.callbacks.manager [None req-55e237ff-69a1-4c17-bc65-40d9cc35423a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8760688960988', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-7783166'] for agent, after_update {{(pid=91645) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 20 04:43:01.054113 dv-cinder-782405-9-iscsi neutron-server[91646]: DEBUG neutron.db.agents_db [None req-6923152c-f506-4748-9493-2c5311d49ad6 None None] Agent healthcheck: found 6 active agents {{(pid=91646) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:325}}